-- Logs begin at Fri 2022-06-24 23:05:53 UTC, end at Sat 2022-06-25 00:11:46 UTC. -- Jun 24 23:34:35.022982 dv-cinder-847624-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 23:34:35.031412 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 23:34:35 2022] *** Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: nodename: dv-cinder-847624-1-iscsi Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: machine: x86_64 Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: clock source: unix Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: pcre jit disabled Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: detected number of CPU cores: 4 Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: current working directory: / Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: detected binary path: /usr/bin/uwsgi-core Jun 24 23:34:35.033420 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: your processes number limit is 47680 Jun 24 23:34:35.034039 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: your memory page size is 4096 bytes Jun 24 23:34:35.034039 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: detected max file descriptor number: 2048 Jun 24 23:34:35.034039 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: lock engine: pthread robust mutexes Jun 24 23:34:35.034039 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: thunder lock: enabled Jun 24 23:34:35.034039 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 23:34:35.034039 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 23:34:35.077312 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: Python main interpreter initialized at 0x56478065d9c0 Jun 24 23:34:35.077312 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: python threads support enabled Jun 24 23:34:35.077312 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: your server socket listen backlog is limited to 100 connections Jun 24 23:34:35.077312 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: your mercy for graceful operations on workers is 90 seconds Jun 24 23:34:35.077627 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 23:34:35.077722 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: *** Operational MODE: preforking *** Jun 24 23:34:35.077989 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: *** uWSGI is running in multiple interpreter mode *** Jun 24 23:34:35.077989 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: spawned uWSGI master process (pid: 86599) Jun 24 23:34:35.078174 dv-cinder-847624-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 23:34:35.078989 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: spawned uWSGI worker 1 (pid: 86604, cores: 1) Jun 24 23:34:35.079087 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: spawned uWSGI worker 2 (pid: 86605, cores: 1) Jun 24 23:34:35.079226 dv-cinder-847624-1-iscsi devstack@keystone.service[86599]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 23:34:40.580434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 23:34:40.580434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: from cryptography.utils import int_from_bytes Jun 24 23:34:40.581255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 23:34:40.581255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: from cryptography.utils import int_from_bytes Jun 24 23:34:40.582121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 23:34:40.582121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: from cryptography.utils import int_from_bytes Jun 24 23:34:40.582580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 23:34:40.582580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: from cryptography.utils import int_from_bytes Jun 24 23:34:40.630612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 23:34:40.630810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 23:34:40.630985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] command line args: [] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 23:34:40.631260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 23:34:40.631442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ================================================================================ {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 23:34:40.631558 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 23:34:40.631708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] admin_token = **** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.631785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 23:34:40.631935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] command line args: [] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 23:34:40.632017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] config_dir = [] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.632135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.632216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 23:34:40.632320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] config_source = [] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.632396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ================================================================================ {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 23:34:40.632500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] debug = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.632624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] admin_token = **** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.632724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] config_dir = [] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] config_source = [] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] debug = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.633923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634033 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] list_limit = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] log_config_append = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] log_dir = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] log_file = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] list_limit = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] log_config_append = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.634981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] log_dir = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] log_file = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635548 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.635942 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.636021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.636138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.636217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.636339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.636421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.636515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.636630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.636812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.636900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.637051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.637169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.637288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.637363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.637541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.637626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.637755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.637828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.637984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.638060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.638230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.638475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.638551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.638701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] publish_errors = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.638783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.638940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.639174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.639254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.639355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.639438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] publish_errors = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.639598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.639678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.639826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.639907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.639993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.640116 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] standard_threads = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.640232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.640309 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.640464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.640545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.640712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.640797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] standard_threads = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.640894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] use_journal = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.640976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641073 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] use_json = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] use_stderr = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] use_syslog = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] use_journal = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] use_json = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] use_stderr = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.641918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.642018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] use_syslog = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.642127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.642245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 23:34:40.642392 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.642498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.642635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.642734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.642909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.643032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.643164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.643293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.643417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.643492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.643657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.643739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.643907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.643984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.644158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.644240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.644411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.644485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.644671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.644750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.644921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.645005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.645154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.645281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.645399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.645481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.645633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.645708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.645874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.645956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.646107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.646183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.646343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.646419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.646595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.646670 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.646837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.646919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.647334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.647416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.647580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.647655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.647830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.647913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.648086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.648160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.648321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.648397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.648554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.648658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.648811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.648888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.649037 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.649124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.649266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.649342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.649499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.649574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.649743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.649825 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.649976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.650052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.650200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.650273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.650427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.650511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.650670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.650746 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.650915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.650990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.651148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.651231 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.651387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.651462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.651613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.651687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.651846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.651998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.652076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.652146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.652290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.652366 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.652517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.652662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.652769 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.652852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.652999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.653075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.653233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.653317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.653471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.653546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.653697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.653773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.653934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.654042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.654160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.654261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.654395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.654497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.654630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.654739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.654867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.654944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.655093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.655170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.655314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.655399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.655550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.655628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.655784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.655860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.656017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.656101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.656249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.656324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.656478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.656554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.656743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.656822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.656977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.657054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.657212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.657301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.657439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.657524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.657678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.657761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.657916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.657994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.658142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.658221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.658378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.658463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.658665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.658780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.658934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.659047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.659170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.659385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.659461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.659632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.659715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.659864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.659948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.660097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.660329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.660405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.660574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.660679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.660839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.660915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.661072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.661147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.661313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.661397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.661548 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.661625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.661782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.661864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.662017 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.662100 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.662251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.662333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.662486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.662573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.662746 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.662829 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.662983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.663059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.663223 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.663299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.663458 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.663613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.663687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.663755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.663920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.663996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.664154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.664239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.664386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.664473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.664630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.664711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.664863 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.664938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.665091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.665183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.665322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.665398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.665546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.665627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.665782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.665869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.666023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.666099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.666261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.666336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.666493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.666576 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.666745 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.666822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.667001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.667075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.667238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] auth.external = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.667315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.667463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] auth.external = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.667534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.667693 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.667765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.667984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.668069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.668210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.668287 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] auth.password = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.668439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] auth.password = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.668515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] auth.token = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.668700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.668776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] auth.token = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.668937 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.669023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.669171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.669249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.669407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.669481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.669632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.669709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.669876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.669960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.670113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.670188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.670338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.670421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] credential.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.670572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] credential.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.670648 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.670815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.670892 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.671062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.671141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.671301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.671376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.671533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.671608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.671772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.671851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.672005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.672082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.672269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.672346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.672509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.672616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.672768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.672857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.673021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.673099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.673298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.673375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.673538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.673620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.673799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.673883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.674040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.674117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.674810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] federation.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.674887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.675046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] federation.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.675176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.675284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.675359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.675518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.675596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.675751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.675826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.675979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.676056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.676227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.676304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.676471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.676547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.676734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.676813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.676987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.677063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.677225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.677302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.677469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.677547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.677707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.677783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.677940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.678016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.678185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.678265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.678423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.678499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.678656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.678740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.678891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.678969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.679121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.679197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.679353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.679442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.679589 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.679666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.679835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.679912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.680066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.680150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.680299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.680375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.680528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.680624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.680784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.680873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.681028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.681105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.681272 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.681421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.681511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.681595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.681755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.681833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.681999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.682076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.682255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.682339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.682488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.682564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.682720 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.682796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.682948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.683032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.683180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.683256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.683404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.683481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.683638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.683731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.683877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.683958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.684114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.684192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.684344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.684428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.684571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.684668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.684838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.684915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.685082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.685174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.685314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.685397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.685556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.685634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.685787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.685871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.686012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.686088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.686245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.686321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.686476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.686562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.686711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.686789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.686943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.687020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.687179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.687263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.687410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.687486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.687644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.687727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.687888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.687974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.688120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.688195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.688354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.688431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.688658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.688752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.688848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.688926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.689094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.689171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.689329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.689415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.689564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.689642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.689841 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.690423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.690596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.690729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.690859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.691089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.691172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.691244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.691324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.691396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.691567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.691652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.691823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.691901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.692063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.692143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.692304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.692420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.692526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.692619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.692789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.692870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.693025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.693104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.693255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.693338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.693480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.693558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.693725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.693803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.693969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.694046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.694213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.694292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.694456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.694534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.694686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.694762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.694916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.694995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.695159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.695236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.695394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.695474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.695627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.695706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.695860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.695938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.696092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.696169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.696334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.696413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.696596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.696675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.696839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.696916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.697078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.697157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.697308 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.697385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.697550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.697628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.697784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.697863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.698017 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.698095 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.698252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.698329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.698495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.698573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.698730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.698807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.698957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.699042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.699191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.699270 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] resource.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.699429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] resource.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.699506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.699676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.699751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.699916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.699994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.700145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.700222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.700392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.700470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.700665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.700748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.700913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.700991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.701153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.701229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.701380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.701454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.701615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.701693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] role.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.701850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] role.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.701927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] role.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.702091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.702168 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] role.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.702318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.702395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.702566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.702643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.702813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.702892 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.703059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.703136 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.703304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.703390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.703549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.703627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.703794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.703871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.704035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.704119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.704269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.704345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.704497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.704575 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.704774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.704863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.705020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.705099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.705265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.705343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.705599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.705801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.705930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.706007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.706187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.706265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.706438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.706523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.706688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.706764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.706929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.707004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.707165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.707249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.707397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.707473 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.707648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.707726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.707893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.707977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.708128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.708204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.708359 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.708436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.708597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.708689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.708842 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.708920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.709091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.709168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.709343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.709420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.709586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.709661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.709824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.709900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.710056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.710133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.710284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.710361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] token.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.710519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] token.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.710595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.710755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.710834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.710993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.711068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.711224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.711300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.711477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.711555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.711721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.711796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.711954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.712030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.712196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.712274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.712438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.712515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.712697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.712775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.712933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.713012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.713169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.713244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.713400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.713475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.713643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.713723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.713888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.713963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.714112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.714187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.714354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.714433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.714640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.714722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.connection = **** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.714902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.714987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.connection = **** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.715145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.715229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.715384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.715468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.715622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.715698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.715864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.715940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.716099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.716184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.716335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.716410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.716572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.716665 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.716833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.716922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.717069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.717145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.717300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.717376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.717542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.717629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.717772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.717848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.718017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.718092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.718246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.718331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.718484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.718560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.718708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.718783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.718947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.719031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.719104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 23:34:40.719171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 23:34:40.719334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 23:34:40.750144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.750373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.750474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.750601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.750683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.750844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.759286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.759525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.788486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.788628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.788740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.788822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.932865 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.937151 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.984858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.985191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.986216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:40.986458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.051809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.052033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.052260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.052483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.052718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.052807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.053010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.053090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.053396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.053681 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.053965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.054244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.078279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.078405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.078551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.078720 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.078801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.078941 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.079158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.079246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.079373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.079547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.079627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.079874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.080155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.081162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.091446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.091893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.104179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.104414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.104679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.104906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.105126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.105208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.105344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.105563 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.105645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.105893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.106194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.106476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.106754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.107033 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 23:34:41.122857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.123072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 23:34:41.123412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.123492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.123615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 23:34:41.123810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 23:34:41.123891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.124131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.124313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.124394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.124647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 23:34:41.124731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.125023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.125348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.125666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.125983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.136094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.136376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.136672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.136960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.138078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.138421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.138753 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.139099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.145824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.148966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.149410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.149695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.149982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.150264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.150542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.150801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.151065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.151337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.151605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.151876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.152142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.153013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.153360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.153706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.154063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.154409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.154735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.155066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.155401 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.155737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.156074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.156406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.179609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.181379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.181663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.184510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.186411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.186732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.186932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.187324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.187647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.192628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.193022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.193306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.195364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.195650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.200154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.200442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.200571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.200751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.200879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.201040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.201317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.201594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.201865 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.202130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.205335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.206576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.206865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.207152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.207432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.207706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.207975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.208244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.221459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.221737 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.226387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.227440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.227715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.230064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.230471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.232312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.235510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.235998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.236398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.239675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.242435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.243347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.243659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.243948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.244244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.244526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.244901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.246033 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.249706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.250005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.250290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.250575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.250853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.251127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.264721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.265031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.265304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.265582 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.272892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.273193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.273464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.273739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.274029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.276588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.276891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.281478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.281656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.281771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.282057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.282346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.282636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.284225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.284510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.287759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.288047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.288332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.288630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.288914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.296027 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.296310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.296626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.296915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.298631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.298925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.299801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.302765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.303048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.303337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.303616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.305357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.305654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.305951 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.318362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.321707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.324449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.325024 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.326762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.327752 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.328406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.328714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.328998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.331262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.333025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.334656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.334934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.335222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.336418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.338059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.341358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.341653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.342049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.342330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.342454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.344092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.347614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.347908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.348294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.348571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.351581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.351870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.352153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.357673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.357961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.358242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.360043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.360323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.360627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.360910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.361184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.361572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.362394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.362673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.362949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.363240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.363514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.363777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.364044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.364324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.366683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 23:34:41.366962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.367242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.367518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.367791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.368174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.368451 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.368756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.369032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.369323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.369596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.369859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.370135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.370405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 23:34:41.403341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56478065d9c0 pid: 86604 (default app) Jun 24 23:34:41.406725 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b375db2-625c-44b2-bee2-4daddb3fc60e None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:41.407034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b375db2-625c-44b2-bee2-4daddb3fc60e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:41.407318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b375db2-625c-44b2-bee2-4daddb3fc60e None None] PATH_INFO: `/v3` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:41.409068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56478065d9c0 pid: 86605 (default app) Jun 24 23:34:41.411856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Fri Jun 24 23:34:41 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:34:49.066713 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03480d2f-f11d-4252-b6fe-4c5ffa90b2a1 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:49.067274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03480d2f-f11d-4252-b6fe-4c5ffa90b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:49.067274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03480d2f-f11d-4252-b6fe-4c5ffa90b2a1 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:49.071761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:34:49.077159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:49.077545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:49.077722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:49.148167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: INFO dbcounter [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] Registered counter for database keystone Jun 24 23:34:49.169489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG oslo_db.sqlalchemy.engines [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 23:34:49.176221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writer thread running {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 23:34:49.358770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG passlib.handlers.bcrypt [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86604) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 23:34:49.359189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG passlib.handlers.bcrypt [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86604) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 23:34:49.414357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:49.415964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:49.537795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 23:34:49.688759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03480d2f-f11d-4252-b6fe-4c5ffa90b2a1 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:49.761406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: INFO dbcounter [None req-03480d2f-f11d-4252-b6fe-4c5ffa90b2a1 None None] Registered counter for database keystone Jun 24 23:34:49.781026 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG oslo_db.sqlalchemy.engines [None req-03480d2f-f11d-4252-b6fe-4c5ffa90b2a1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 23:34:49.788196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writer thread running {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 23:34:49.913475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03480d2f-f11d-4252-b6fe-4c5ffa90b2a1 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:49.917775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628483b-f3e8-4e87-978b-7547ae301a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:49.918440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628483b-f3e8-4e87-978b-7547ae301a50 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:49.918726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628483b-f3e8-4e87-978b-7547ae301a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:49.919006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628483b-f3e8-4e87-978b-7547ae301a50 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:49.927729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628483b-f3e8-4e87-978b-7547ae301a50 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:49.980918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a628483b-f3e8-4e87-978b-7547ae301a50 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 23:34:49.981266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628483b-f3e8-4e87-978b-7547ae301a50 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:49.992526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Fri Jun 24 23:34:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:34:49.998458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:50.013445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bf050-9901-4df5-8a2a-cb3565eec41b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:50.018457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed3cbd-ac4f-41cb-a0f3-2707879e3bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:50.019633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed3cbd-ac4f-41cb-a0f3-2707879e3bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:50.019922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed3cbd-ac4f-41cb-a0f3-2707879e3bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:50.020206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed3cbd-ac4f-41cb-a0f3-2707879e3bf1 admin admin] PATH_INFO: `/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:50.024668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed3cbd-ac4f-41cb-a0f3-2707879e3bf1 admin admin] RBAC: Authorizing `identity:get_project(project_id=1b6f1fb3ca4d42909a61ac0a3da08290)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:50.075089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WARNING keystone.common.rbac_enforcer.enforcer [None req-35ed3cbd-ac4f-41cb-a0f3-2707879e3bf1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 23:34:50.075445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed3cbd-ac4f-41cb-a0f3-2707879e3bf1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:50.080106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Fri Jun 24 23:34:49 2022] GET /identity/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:34:52.093987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665baa-7f76-4b3f-9889-9712a19968cc None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:52.094509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665baa-7f76-4b3f-9889-9712a19968cc None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:52.094604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73665baa-7f76-4b3f-9889-9712a19968cc None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:52.099350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:34:52.104623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0aa2-a2c5-4a58-93c3-6b10ead53d2f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:52.105033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0aa2-a2c5-4a58-93c3-6b10ead53d2f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:52.105189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c0aa2-a2c5-4a58-93c3-6b10ead53d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:52.124130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6a8c0aa2-a2c5-4a58-93c3-6b10ead53d2f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:52.125676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6a8c0aa2-a2c5-4a58-93c3-6b10ead53d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:52.133527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 23:34:52.155779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73665baa-7f76-4b3f-9889-9712a19968cc None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:52.168398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73665baa-7f76-4b3f-9889-9712a19968cc None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:52.172663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75412191-aaa1-4e9c-9676-e89ea049355c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:52.173297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75412191-aaa1-4e9c-9676-e89ea049355c admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:52.173583 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75412191-aaa1-4e9c-9676-e89ea049355c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:52.173871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75412191-aaa1-4e9c-9676-e89ea049355c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:52.181819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75412191-aaa1-4e9c-9676-e89ea049355c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:52.183603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75412191-aaa1-4e9c-9676-e89ea049355c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:52.192900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Fri Jun 24 23:34:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:34:52.198067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c0aa2-a2c5-4a58-93c3-6b10ead53d2f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:52.207163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c0aa2-a2c5-4a58-93c3-6b10ead53d2f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:52.211528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4013e4b-f069-46f6-83f6-2c1d042e9748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:52.212322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4013e4b-f069-46f6-83f6-2c1d042e9748 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:52.212630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4013e4b-f069-46f6-83f6-2c1d042e9748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:52.212907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4013e4b-f069-46f6-83f6-2c1d042e9748 admin admin] PATH_INFO: `/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:52.214285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4013e4b-f069-46f6-83f6-2c1d042e9748 admin admin] RBAC: Authorizing `identity:get_user(user_id=19e6b8d9d8f94b0c8f316b8b2060e76d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:52.216087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4013e4b-f069-46f6-83f6-2c1d042e9748 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:52.217673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Fri Jun 24 23:34:52 2022] GET /identity/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:34:54.316450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8a3cb-1e30-4d0a-86ed-a48bb2acec8f None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:54.316993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8a3cb-1e30-4d0a-86ed-a48bb2acec8f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:54.316993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8a3cb-1e30-4d0a-86ed-a48bb2acec8f None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:54.318307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:34:54.323549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934fd9-cf31-4f6d-aaa6-915ecb7210a0 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:54.323961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934fd9-cf31-4f6d-aaa6-915ecb7210a0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:54.324121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934fd9-cf31-4f6d-aaa6-915ecb7210a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:54.341425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-59934fd9-cf31-4f6d-aaa6-915ecb7210a0 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:54.342622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-59934fd9-cf31-4f6d-aaa6-915ecb7210a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:54.355956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5c122-ae4d-46a8-aac6-bfba97e046bc None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:54.356247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5c122-ae4d-46a8-aac6-bfba97e046bc None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:54.356528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5c122-ae4d-46a8-aac6-bfba97e046bc None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:54.357902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:34:54.362928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398c718-d81f-4b0b-9903-df76461cba11 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:54.363208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398c718-d81f-4b0b-9903-df76461cba11 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:54.363483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398c718-d81f-4b0b-9903-df76461cba11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:54.403764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG passlib.handlers.bcrypt [None req-f398c718-d81f-4b0b-9903-df76461cba11 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86605) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 23:34:54.404187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG passlib.handlers.bcrypt [None req-f398c718-d81f-4b0b-9903-df76461cba11 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86605) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 23:34:54.423237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:34:54.446943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934fd9-cf31-4f6d-aaa6-915ecb7210a0 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:54.450488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f398c718-d81f-4b0b-9903-df76461cba11 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:54.452071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f398c718-d81f-4b0b-9903-df76461cba11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:54.470480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:54.552326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934fd9-cf31-4f6d-aaa6-915ecb7210a0 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:54.552935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:34:54.575043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398c718-d81f-4b0b-9903-df76461cba11 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:54.590313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262fca0-286a-471b-bdaa-66133c846275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:54.591120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262fca0-286a-471b-bdaa-66133c846275 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:54.591406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262fca0-286a-471b-bdaa-66133c846275 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:54.591690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262fca0-286a-471b-bdaa-66133c846275 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:54.604072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6262fca0-286a-471b-bdaa-66133c846275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-6262fca0-286a-471b-bdaa-66133c846275 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:34:54.616182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:34:54.617134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:34:54.618070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:34:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:34:54.623076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262fca0-286a-471b-bdaa-66133c846275 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:54.675811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398c718-d81f-4b0b-9903-df76461cba11 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:54.678128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:54.703927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:54.716960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53e45a-8a19-4db5-99db-bed77f1c827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:54.717876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53e45a-8a19-4db5-99db-bed77f1c827c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:54.718168 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53e45a-8a19-4db5-99db-bed77f1c827c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:54.718450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53e45a-8a19-4db5-99db-bed77f1c827c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:54.723469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262fca0-286a-471b-bdaa-66133c846275 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:54.757731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a7465-57ee-4b47-9e3a-d38542259813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:54.758397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a7465-57ee-4b47-9e3a-d38542259813 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:54.758683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a7465-57ee-4b47-9e3a-d38542259813 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:54.758961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a7465-57ee-4b47-9e3a-d38542259813 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:54.759708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a7465-57ee-4b47-9e3a-d38542259813 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:34:54.760014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a7465-57ee-4b47-9e3a-d38542259813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:54.761440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a7465-57ee-4b47-9e3a-d38542259813 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:54.773902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:34:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:34:54.779238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a7465-57ee-4b47-9e3a-d38542259813 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:54.787496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53e45a-8a19-4db5-99db-bed77f1c827c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:54.789053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c53e45a-8a19-4db5-99db-bed77f1c827c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:54.877244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a7465-57ee-4b47-9e3a-d38542259813 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:54.886284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:34:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 23:34:54.891874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53e45a-8a19-4db5-99db-bed77f1c827c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:54.895117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:54.912237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18842a83-d829-407d-a7dc-547616979c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:54.913149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18842a83-d829-407d-a7dc-547616979c9e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:54.913443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18842a83-d829-407d-a7dc-547616979c9e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:54.913721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18842a83-d829-407d-a7dc-547616979c9e None admin] PATH_INFO: `/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:54.914933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18842a83-d829-407d-a7dc-547616979c9e None admin] RBAC: Authorizing `identity:get_user(user_id=19e6b8d9d8f94b0c8f316b8b2060e76d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:54.916397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18842a83-d829-407d-a7dc-547616979c9e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:54.918057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:34:54 2022] GET /identity/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:34:54.923237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18842a83-d829-407d-a7dc-547616979c9e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:54.938056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:54.995132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53e45a-8a19-4db5-99db-bed77f1c827c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:55.023692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18842a83-d829-407d-a7dc-547616979c9e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:55.028471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee8dbf-30fc-4193-90bd-6006b76f0c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:55.029298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee8dbf-30fc-4193-90bd-6006b76f0c85 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:55.029582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee8dbf-30fc-4193-90bd-6006b76f0c85 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:55.029856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee8dbf-30fc-4193-90bd-6006b76f0c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:55.049346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee8dbf-30fc-4193-90bd-6006b76f0c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:55.051061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee8dbf-30fc-4193-90bd-6006b76f0c85 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:55.059775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5e2c5-6e85-4a43-a851-dfbf75fb8f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:55.060574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5e2c5-6e85-4a43-a851-dfbf75fb8f70 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:55.060885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5e2c5-6e85-4a43-a851-dfbf75fb8f70 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:55.061161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5e2c5-6e85-4a43-a851-dfbf75fb8f70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:55.062345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5e2c5-6e85-4a43-a851-dfbf75fb8f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:55.064104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5e2c5-6e85-4a43-a851-dfbf75fb8f70 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:55.066298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 10/18] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:34:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:34:55.071998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5e2c5-6e85-4a43-a851-dfbf75fb8f70 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-96ee8dbf-30fc-4193-90bd-6006b76f0c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:34:55.072198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:34:55.073009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:34:55.073803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:34:55.073803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:34:55.073803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:34:55.073803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:34:55.073803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 9/19] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:34:55.079194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee8dbf-30fc-4193-90bd-6006b76f0c85 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:55.139208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:55.154466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:55.173713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5e2c5-6e85-4a43-a851-dfbf75fb8f70 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:55.185146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee8dbf-30fc-4193-90bd-6006b76f0c85 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:55.207775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c583e8-8e9b-454d-8b06-a5ea7590d4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:55.208430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c583e8-8e9b-454d-8b06-a5ea7590d4f6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:55.208745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c583e8-8e9b-454d-8b06-a5ea7590d4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:55.209025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c583e8-8e9b-454d-8b06-a5ea7590d4f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:55.209845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c583e8-8e9b-454d-8b06-a5ea7590d4f6 None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.domain.id=default, user.id=19e6b8d9d8f94b0c8f316b8b2060e76d) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:34:55.210155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c583e8-8e9b-454d-8b06-a5ea7590d4f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:55.211596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c583e8-8e9b-454d-8b06-a5ea7590d4f6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:55.218101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 11/20] 10.222.0.47 () {56 vars in 1322 bytes} [Fri Jun 24 23:34:55 2022] GET /identity/v3/role_assignments?user.id=19e6b8d9d8f94b0c8f316b8b2060e76d&scope.domain.id=default&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:34:55.218671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ccd06-8def-47ce-9e1c-b98c7ce5b75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:55.219342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ccd06-8def-47ce-9e1c-b98c7ce5b75c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:55.219635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ccd06-8def-47ce-9e1c-b98c7ce5b75c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:55.219920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ccd06-8def-47ce-9e1c-b98c7ce5b75c None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:55.220686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ccd06-8def-47ce-9e1c-b98c7ce5b75c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:34:55.221566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ccd06-8def-47ce-9e1c-b98c7ce5b75c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:55.222974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ccd06-8def-47ce-9e1c-b98c7ce5b75c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:55.243810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 10/21] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:34:57.570712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38decd-55b1-4f84-b11d-e512112fb63d None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:57.571217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38decd-55b1-4f84-b11d-e512112fb63d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:57.571314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38decd-55b1-4f84-b11d-e512112fb63d None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:57.572636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 12/22] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:34:57.578297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7299-4077-4ce1-a6d6-1a0fc0c0a50e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:57.578716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7299-4077-4ce1-a6d6-1a0fc0c0a50e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:57.578896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db7299-4077-4ce1-a6d6-1a0fc0c0a50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:57.603404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e901c5-53dc-4880-9d7a-e533569fd9bb None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:57.603714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e901c5-53dc-4880-9d7a-e533569fd9bb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:57.603968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e901c5-53dc-4880-9d7a-e533569fd9bb None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:57.605853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 13/23] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:34:57.605956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-93db7299-4077-4ce1-a6d6-1a0fc0c0a50e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:57.607451 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-93db7299-4077-4ce1-a6d6-1a0fc0c0a50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:57.610777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99010c-e499-42ca-a1ef-37a23577dfb1 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:57.611071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99010c-e499-42ca-a1ef-37a23577dfb1 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:57.611430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99010c-e499-42ca-a1ef-37a23577dfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:57.638421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6a99010c-e499-42ca-a1ef-37a23577dfb1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:57.640024 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6a99010c-e499-42ca-a1ef-37a23577dfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:57.754039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 11/24] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:34:57.789042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db7299-4077-4ce1-a6d6-1a0fc0c0a50e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:57.798449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 14/25] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:34:57.839521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99010c-e499-42ca-a1ef-37a23577dfb1 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:57.870114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:57.987926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db7299-4077-4ce1-a6d6-1a0fc0c0a50e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:58.029294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496e50f-9bbe-42a0-be14-7aa8ad070b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:58.030456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496e50f-9bbe-42a0-be14-7aa8ad070b00 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.031191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496e50f-9bbe-42a0-be14-7aa8ad070b00 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.032670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496e50f-9bbe-42a0-be14-7aa8ad070b00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.044555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:58.057218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496e50f-9bbe-42a0-be14-7aa8ad070b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-d496e50f-9bbe-42a0-be14-7aa8ad070b00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:34:58.074413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:34:58.075332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:34:58.076071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 12/26] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:34:58.077221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d570f90-76ad-4fcb-9bfe-04004cbc7e52 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.077680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d570f90-76ad-4fcb-9bfe-04004cbc7e52 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.077790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99010c-e499-42ca-a1ef-37a23577dfb1 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:58.078190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d570f90-76ad-4fcb-9bfe-04004cbc7e52 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.079739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 13/27] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:34:58.085948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d570f90-76ad-4fcb-9bfe-04004cbc7e52 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:58.124369 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8945e-d2f3-4a48-beb1-6adbbb2bbd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:58.125605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8945e-d2f3-4a48-beb1-6adbbb2bbd13 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.126131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8945e-d2f3-4a48-beb1-6adbbb2bbd13 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.126633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8945e-d2f3-4a48-beb1-6adbbb2bbd13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.150900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8945e-d2f3-4a48-beb1-6adbbb2bbd13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:58.153535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd8945e-d2f3-4a48-beb1-6adbbb2bbd13 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-4dd8945e-d2f3-4a48-beb1-6adbbb2bbd13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:34:58.190469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:34:58.191326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:34:58.192101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:34:58.192101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:34:58.192101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:34:58.192101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:34:58.201949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 15/28] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:34:58.209788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059338b5-2003-45ad-9a3a-36e60499daee None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.210321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059338b5-2003-45ad-9a3a-36e60499daee None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.210816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059338b5-2003-45ad-9a3a-36e60499daee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.245476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-059338b5-2003-45ad-9a3a-36e60499daee None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:58.253356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-059338b5-2003-45ad-9a3a-36e60499daee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:58.325908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d570f90-76ad-4fcb-9bfe-04004cbc7e52 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:58.330069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 16/29] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:34:58.331454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059338b5-2003-45ad-9a3a-36e60499daee None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:58.341539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:58.378763 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526b00e-d0c8-42d1-b583-c10cfb5f2c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:58.379363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526b00e-d0c8-42d1-b583-c10cfb5f2c89 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.379651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526b00e-d0c8-42d1-b583-c10cfb5f2c89 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.379930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526b00e-d0c8-42d1-b583-c10cfb5f2c89 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.380715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526b00e-d0c8-42d1-b583-c10cfb5f2c89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:34:58.381028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526b00e-d0c8-42d1-b583-c10cfb5f2c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:58.382660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526b00e-d0c8-42d1-b583-c10cfb5f2c89 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:58.394544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 14/30] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:34:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:34:58.400762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed83f3b-1be1-4a74-ac46-cf5645e7274f None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.401061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed83f3b-1be1-4a74-ac46-cf5645e7274f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.401350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed83f3b-1be1-4a74-ac46-cf5645e7274f None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.402011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:58.402756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 15/31] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:34:58.403377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed83f3b-1be1-4a74-ac46-cf5645e7274f None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:58.460141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059338b5-2003-45ad-9a3a-36e60499daee None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:58.507581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a313e-f666-4ce9-abad-408086276177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:58.508912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a313e-f666-4ce9-abad-408086276177 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.509569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a313e-f666-4ce9-abad-408086276177 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.510051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a313e-f666-4ce9-abad-408086276177 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.511554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a313e-f666-4ce9-abad-408086276177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:34:58.512089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a313e-f666-4ce9-abad-408086276177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:58.514454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a313e-f666-4ce9-abad-408086276177 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:58.516982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed83f3b-1be1-4a74-ac46-cf5645e7274f None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:58.537762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 17/32] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:34:58.538613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a313e-f666-4ce9-abad-408086276177 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:58.561489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7108b7-c54f-48a5-bd75-aa755a088973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:58.562782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7108b7-c54f-48a5-bd75-aa755a088973 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.563270 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7108b7-c54f-48a5-bd75-aa755a088973 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.563993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7108b7-c54f-48a5-bd75-aa755a088973 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.565450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7108b7-c54f-48a5-bd75-aa755a088973 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:58.567082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7108b7-c54f-48a5-bd75-aa755a088973 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:58.658710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a313e-f666-4ce9-abad-408086276177 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:58.678875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:58.698888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de458cff-0eb4-421e-899c-ff67cdb69b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:58.699774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de458cff-0eb4-421e-899c-ff67cdb69b9a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.700159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de458cff-0eb4-421e-899c-ff67cdb69b9a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.700447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de458cff-0eb4-421e-899c-ff67cdb69b9a None admin] PATH_INFO: `/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.701700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de458cff-0eb4-421e-899c-ff67cdb69b9a None admin] RBAC: Authorizing `identity:get_user(user_id=19e6b8d9d8f94b0c8f316b8b2060e76d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:58.703266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de458cff-0eb4-421e-899c-ff67cdb69b9a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:58.710262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 18/33] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:34:58 2022] GET /identity/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:34:58.711554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54bd41-428b-48fc-9a94-1523dd370341 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:58.711904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54bd41-428b-48fc-9a94-1523dd370341 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:58.712191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54bd41-428b-48fc-9a94-1523dd370341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:58.733397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3b54bd41-428b-48fc-9a94-1523dd370341 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:58.734938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3b54bd41-428b-48fc-9a94-1523dd370341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:58.741044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 16/34] 10.222.0.47 () {60 vars in 1131 bytes} [Fri Jun 24 23:34:58 2022] POST /identity/v3/roles => generated 225 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:34:58.742183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7108b7-c54f-48a5-bd75-aa755a088973 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:58.806458 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:58.873237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 19/35] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:34:58.875163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54bd41-428b-48fc-9a94-1523dd370341 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:58.980906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7108b7-c54f-48a5-bd75-aa755a088973 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:59.016959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54bd41-428b-48fc-9a94-1523dd370341 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:59.022943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10becc9-4e87-4d53-9e5b-c60978c6c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:59.023934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10becc9-4e87-4d53-9e5b-c60978c6c9a0 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.024433 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10becc9-4e87-4d53-9e5b-c60978c6c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.024955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10becc9-4e87-4d53-9e5b-c60978c6c9a0 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.026254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10becc9-4e87-4d53-9e5b-c60978c6c9a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:59.028194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10becc9-4e87-4d53-9e5b-c60978c6c9a0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:59.032970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:59.058176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a6ea5-44a2-4b3e-8d13-a3d0d0c1adad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:59.059158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a6ea5-44a2-4b3e-8d13-a3d0d0c1adad None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.059602 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a6ea5-44a2-4b3e-8d13-a3d0d0c1adad None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.060090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a6ea5-44a2-4b3e-8d13-a3d0d0c1adad None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.061752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a6ea5-44a2-4b3e-8d13-a3d0d0c1adad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:59.064080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a6ea5-44a2-4b3e-8d13-a3d0d0c1adad None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:59.066463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 20/36] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:34:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:34:59.067706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a6ea5-44a2-4b3e-8d13-a3d0d0c1adad None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:59.071609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 17/37] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:34:58 2022] POST /identity/v3/projects => generated 301 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:34:59.072378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10becc9-4e87-4d53-9e5b-c60978c6c9a0 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:59.190058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:59.213361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10becc9-4e87-4d53-9e5b-c60978c6c9a0 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:59.213568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a6ea5-44a2-4b3e-8d13-a3d0d0c1adad None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:59.252988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9bb96-2c59-4a09-9c15-8694d8fe6485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:59.253199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dfb06-9a3d-464e-8a50-ed55d27b8fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:59.253678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9bb96-2c59-4a09-9c15-8694d8fe6485 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.254077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dfb06-9a3d-464e-8a50-ed55d27b8fb0 None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.254371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dfb06-9a3d-464e-8a50-ed55d27b8fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.254620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9bb96-2c59-4a09-9c15-8694d8fe6485 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.254699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dfb06-9a3d-464e-8a50-ed55d27b8fb0 None admin] PATH_INFO: `/v3/domains/default/users/19e6b8d9d8f94b0c8f316b8b2060e76d/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.254946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9bb96-2c59-4a09-9c15-8694d8fe6485 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.255731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9bb96-2c59-4a09-9c15-8694d8fe6485 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:59.256995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73dfb06-9a3d-464e-8a50-ed55d27b8fb0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=19e6b8d9d8f94b0c8f316b8b2060e76d, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:59.257196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9bb96-2c59-4a09-9c15-8694d8fe6485 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:59.258449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73dfb06-9a3d-464e-8a50-ed55d27b8fb0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:59.278726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 18/38] 10.222.0.47 () {58 vars in 1361 bytes} [Fri Jun 24 23:34:59 2022] PUT /identity/v3/domains/default/users/19e6b8d9d8f94b0c8f316b8b2060e76d/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:34:59.280137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbe94a-35ca-4fb8-9ab3-b6541745aaff None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.280436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbe94a-35ca-4fb8-9ab3-b6541745aaff None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.280797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbe94a-35ca-4fb8-9ab3-b6541745aaff None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.282135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 19/39] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:34:59.283909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecf1d4-b4ca-4475-97dd-bfdaa267f963 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.284301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecf1d4-b4ca-4475-97dd-bfdaa267f963 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.284483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecf1d4-b4ca-4475-97dd-bfdaa267f963 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.287289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 20/40] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:34:59.292599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a71a08-00ef-4eb5-86bb-f4e0e60ace80 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.292904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a71a08-00ef-4eb5-86bb-f4e0e60ace80 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.293226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a71a08-00ef-4eb5-86bb-f4e0e60ace80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.315602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-e6a71a08-00ef-4eb5-86bb-f4e0e60ace80 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:59.317242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e6a71a08-00ef-4eb5-86bb-f4e0e60ace80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:59.391463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 21/41] 10.222.0.47 () {60 vars in 1131 bytes} [Fri Jun 24 23:34:59 2022] POST /identity/v3/roles => generated 219 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:34:59.393534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced060fa-6794-4319-9fd5-380927222d31 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.394385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced060fa-6794-4319-9fd5-380927222d31 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.395500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced060fa-6794-4319-9fd5-380927222d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.414587 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 21/42] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:34:59.439677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a71a08-00ef-4eb5-86bb-f4e0e60ace80 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:59.530222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:59.533419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ced060fa-6794-4319-9fd5-380927222d31 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:34:59.535831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ced060fa-6794-4319-9fd5-380927222d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:34:59.583512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:59.583701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a71a08-00ef-4eb5-86bb-f4e0e60ace80 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:59.623032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 22/43] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:34:59.633675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b45dbd-6d2f-46aa-94b8-bfb79f2fa64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:59.634595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b45dbd-6d2f-46aa-94b8-bfb79f2fa64e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.634959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b45dbd-6d2f-46aa-94b8-bfb79f2fa64e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.635318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b45dbd-6d2f-46aa-94b8-bfb79f2fa64e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.636637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b45dbd-6d2f-46aa-94b8-bfb79f2fa64e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:59.638479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b45dbd-6d2f-46aa-94b8-bfb79f2fa64e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:59.640117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 22/44] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:34:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:34:59.646151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced060fa-6794-4319-9fd5-380927222d31 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:59.649280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b45dbd-6d2f-46aa-94b8-bfb79f2fa64e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:34:59.765816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced060fa-6794-4319-9fd5-380927222d31 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:59.785986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b45dbd-6d2f-46aa-94b8-bfb79f2fa64e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:34:59.802795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f89aa-34f8-4501-957a-afa9ecfc0fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:59.803473 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f89aa-34f8-4501-957a-afa9ecfc0fe0 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.803767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f89aa-34f8-4501-957a-afa9ecfc0fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.804053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f89aa-34f8-4501-957a-afa9ecfc0fe0 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.804938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f89aa-34f8-4501-957a-afa9ecfc0fe0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:59.806686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f89aa-34f8-4501-957a-afa9ecfc0fe0 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:59.822424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d4fdd-ffbc-4fc4-9b66-503a0448030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:34:59.823103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d4fdd-ffbc-4fc4-9b66-503a0448030f None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:34:59.823399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d4fdd-ffbc-4fc4-9b66-503a0448030f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:34:59.823685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d4fdd-ffbc-4fc4-9b66-503a0448030f None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:34:59.824457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d4fdd-ffbc-4fc4-9b66-503a0448030f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:34:59.826286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d4fdd-ffbc-4fc4-9b66-503a0448030f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:34:59.834366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:34:59.848613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 23/45] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:34:59 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:34:59.848838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 23/46] 10.222.0.47 () {60 vars in 1131 bytes} [Fri Jun 24 23:34:59 2022] POST /identity/v3/roles => generated 223 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:35:01.384133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae564-adf4-409c-9c7e-a2c03112d90f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:01.384665 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae564-adf4-409c-9c7e-a2c03112d90f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:01.384747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beae564-adf4-409c-9c7e-a2c03112d90f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:01.386140 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 24/47] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:01.391346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05383cb6-5051-434c-bc51-04d4bee1bdeb None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:01.391765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05383cb6-5051-434c-bc51-04d4bee1bdeb None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:01.391912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05383cb6-5051-434c-bc51-04d4bee1bdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:01.409604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-05383cb6-5051-434c-bc51-04d4bee1bdeb None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:01.410804 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-05383cb6-5051-434c-bc51-04d4bee1bdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:01.484425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 24/48] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:01.508398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beae564-adf4-409c-9c7e-a2c03112d90f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:01.534264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:01.612013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beae564-adf4-409c-9c7e-a2c03112d90f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:01.645283 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c12fc2-70e6-447c-b802-79a1dffea8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:01.646017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c12fc2-70e6-447c-b802-79a1dffea8cb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:01.646311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c12fc2-70e6-447c-b802-79a1dffea8cb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:01.646594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c12fc2-70e6-447c-b802-79a1dffea8cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:01.657487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c12fc2-70e6-447c-b802-79a1dffea8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-91c12fc2-70e6-447c-b802-79a1dffea8cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:01.665268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:01.666237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:01.667083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 25/49] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:01.672130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05383cb6-5051-434c-bc51-04d4bee1bdeb None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:01.785183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05383cb6-5051-434c-bc51-04d4bee1bdeb None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:01.811508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:01.820190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d00fac-c772-4865-b3cf-f689f11f8765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:01.820895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d00fac-c772-4865-b3cf-f689f11f8765 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:01.821185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d00fac-c772-4865-b3cf-f689f11f8765 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:01.821466 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d00fac-c772-4865-b3cf-f689f11f8765 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:01.822227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d00fac-c772-4865-b3cf-f689f11f8765 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:01.822542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d00fac-c772-4865-b3cf-f689f11f8765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:01.823927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d00fac-c772-4865-b3cf-f689f11f8765 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:01.832466 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 25/50] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:01.837858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c12fc2-70e6-447c-b802-79a1dffea8cb None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:01.901227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8a5e5-3c2a-4f55-b47d-32224bb68353 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:01.901432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8a5e5-3c2a-4f55-b47d-32224bb68353 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:01.901708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8a5e5-3c2a-4f55-b47d-32224bb68353 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:01.903070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 26/51] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:35:01.907760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bca4e-9873-4cf3-8fae-3d6da1106a1d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:01.908054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bca4e-9873-4cf3-8fae-3d6da1106a1d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:01.908850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bca4e-9873-4cf3-8fae-3d6da1106a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:01.922323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:01.925680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f32bca4e-9873-4cf3-8fae-3d6da1106a1d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:01.927110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f32bca4e-9873-4cf3-8fae-3d6da1106a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:01.940839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c12fc2-70e6-447c-b802-79a1dffea8cb None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:01.977533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0585d-e111-4fc9-b8cb-60cdcda3435b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:01.978417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0585d-e111-4fc9-b8cb-60cdcda3435b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:01.978698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0585d-e111-4fc9-b8cb-60cdcda3435b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:01.978976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0585d-e111-4fc9-b8cb-60cdcda3435b None admin] PATH_INFO: `/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:01.980177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0585d-e111-4fc9-b8cb-60cdcda3435b None admin] RBAC: Authorizing `identity:get_user(user_id=19e6b8d9d8f94b0c8f316b8b2060e76d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:01.981684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0585d-e111-4fc9-b8cb-60cdcda3435b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:01.983324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 26/52] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:01 2022] GET /identity/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:01.989488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0585d-e111-4fc9-b8cb-60cdcda3435b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:01.996782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 27/53] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:02.019344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bca4e-9873-4cf3-8fae-3d6da1106a1d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:02.089768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0585d-e111-4fc9-b8cb-60cdcda3435b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:02.125596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3902548-9869-4d0f-a4cc-1d6cb2f6a8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:02.126415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902548-9869-4d0f-a4cc-1d6cb2f6a8d8 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:02.126703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902548-9869-4d0f-a4cc-1d6cb2f6a8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:02.127643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3902548-9869-4d0f-a4cc-1d6cb2f6a8d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:02.128937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3902548-9869-4d0f-a4cc-1d6cb2f6a8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:02.130435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3902548-9869-4d0f-a4cc-1d6cb2f6a8d8 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:02.132105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 27/54] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:02.138052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3902548-9869-4d0f-a4cc-1d6cb2f6a8d8 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:02.162151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:02.221986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bca4e-9873-4cf3-8fae-3d6da1106a1d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:02.241459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3902548-9869-4d0f-a4cc-1d6cb2f6a8d8 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:02.242247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:02.257140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4ad37-0979-4b6b-9115-472fcfe76aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:02.257937 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4ad37-0979-4b6b-9115-472fcfe76aef None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:02.258231 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4ad37-0979-4b6b-9115-472fcfe76aef None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:02.258513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4ad37-0979-4b6b-9115-472fcfe76aef None admin] PATH_INFO: `/v3/domains/default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:02.259701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4ad37-0979-4b6b-9115-472fcfe76aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:02.261187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4ad37-0979-4b6b-9115-472fcfe76aef None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:02.262823 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 28/55] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:02.268671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4ad37-0979-4b6b-9115-472fcfe76aef None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:02.276201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae43ffc-276b-4fa1-b857-52a1aeb2d741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:02.276900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43ffc-276b-4fa1-b857-52a1aeb2d741 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:02.277185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43ffc-276b-4fa1-b857-52a1aeb2d741 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:02.277458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43ffc-276b-4fa1-b857-52a1aeb2d741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:02.278286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43ffc-276b-4fa1-b857-52a1aeb2d741 None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.domain.id=default, user.id=19e6b8d9d8f94b0c8f316b8b2060e76d) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:02.278599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43ffc-276b-4fa1-b857-52a1aeb2d741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:02.280052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43ffc-276b-4fa1-b857-52a1aeb2d741 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:02.285833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 28/56] 10.222.0.47 () {56 vars in 1322 bytes} [Fri Jun 24 23:35:02 2022] GET /identity/v3/role_assignments?user.id=19e6b8d9d8f94b0c8f316b8b2060e76d&scope.domain.id=default&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:02.373384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4ad37-0979-4b6b-9115-472fcfe76aef None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:02.409748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d9a28-0133-4c8e-ac37-1ce3a8be64e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:02.410421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d9a28-0133-4c8e-ac37-1ce3a8be64e0 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:02.410701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d9a28-0133-4c8e-ac37-1ce3a8be64e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:02.410981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d9a28-0133-4c8e-ac37-1ce3a8be64e0 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:02.411715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d9a28-0133-4c8e-ac37-1ce3a8be64e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:02.413288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d9a28-0133-4c8e-ac37-1ce3a8be64e0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:02.439749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 29/57] 10.222.0.47 () {60 vars in 1140 bytes} [Fri Jun 24 23:35:02 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:35:04.456735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d8eb5-cb95-41ae-b137-b4a2be0f30a3 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:04.457243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d8eb5-cb95-41ae-b137-b4a2be0f30a3 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:04.457326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d8eb5-cb95-41ae-b137-b4a2be0f30a3 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:04.458647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 29/58] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:04.463718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc723f9-7b56-4dec-914c-b8571f4aac95 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:04.464119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc723f9-7b56-4dec-914c-b8571f4aac95 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:04.464292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc723f9-7b56-4dec-914c-b8571f4aac95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:04.481704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0cc723f9-7b56-4dec-914c-b8571f4aac95 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:04.482888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0cc723f9-7b56-4dec-914c-b8571f4aac95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:04.515321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:04.549382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 30/59] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:04.572832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d8eb5-cb95-41ae-b137-b4a2be0f30a3 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:04.659101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:04.675625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d8eb5-cb95-41ae-b137-b4a2be0f30a3 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:04.708214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77271e2d-3e4f-439d-b3c3-b6bbd7adbee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:04.708973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77271e2d-3e4f-439d-b3c3-b6bbd7adbee2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:04.709264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77271e2d-3e4f-439d-b3c3-b6bbd7adbee2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:04.709547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77271e2d-3e4f-439d-b3c3-b6bbd7adbee2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:04.710728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77271e2d-3e4f-439d-b3c3-b6bbd7adbee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:04.712180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77271e2d-3e4f-439d-b3c3-b6bbd7adbee2 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:04.713830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 30/60] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:04.719389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc723f9-7b56-4dec-914c-b8571f4aac95 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:04.817904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc723f9-7b56-4dec-914c-b8571f4aac95 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:04.853629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b479e-50fc-4448-a098-352bf3749e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:04.854302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b479e-50fc-4448-a098-352bf3749e8f None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:04.854588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b479e-50fc-4448-a098-352bf3749e8f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:04.854870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b479e-50fc-4448-a098-352bf3749e8f None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:04.855583 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b479e-50fc-4448-a098-352bf3749e8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:04.857068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b479e-50fc-4448-a098-352bf3749e8f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:04.881993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 31/61] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:35:04 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:35:07.981848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2b998-3f80-43d9-9332-17c2731605b5 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:07.982770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2b998-3f80-43d9-9332-17c2731605b5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:07.983235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2b998-3f80-43d9-9332-17c2731605b5 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:07.984861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 31/62] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:07.990744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937c30-f5b7-4f09-b971-e2dd6b9881b8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:07.991170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937c30-f5b7-4f09-b971-e2dd6b9881b8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:07.991315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937c30-f5b7-4f09-b971-e2dd6b9881b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.012684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a5937c30-f5b7-4f09-b971-e2dd6b9881b8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:08.014141 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a5937c30-f5b7-4f09-b971-e2dd6b9881b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:08.044049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:08.105951 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 32/63] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:08.154162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2b998-3f80-43d9-9332-17c2731605b5 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:08.212368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c165b-f0f0-4fdc-a4cf-b68ae2438935 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.212683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c165b-f0f0-4fdc-a4cf-b68ae2438935 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.213026 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c165b-f0f0-4fdc-a4cf-b68ae2438935 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.214819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 33/64] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:08.220613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30514e-0263-42d8-b1e8-c73e134a420f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.221056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30514e-0263-42d8-b1e8-c73e134a420f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.221350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30514e-0263-42d8-b1e8-c73e134a420f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.249654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-2f30514e-0263-42d8-b1e8-c73e134a420f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:08.250880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2f30514e-0263-42d8-b1e8-c73e134a420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:08.331854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2b998-3f80-43d9-9332-17c2731605b5 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:08.360396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 34/65] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:08.363262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a251e8-a970-4ff1-950e-ecd3e24d7c7d None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.363714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a251e8-a970-4ff1-950e-ecd3e24d7c7d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.364135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a251e8-a970-4ff1-950e-ecd3e24d7c7d None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.369614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 35/66] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:08.371603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1342c-13d5-45af-8b76-ea349001e2cf None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.373135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1342c-13d5-45af-8b76-ea349001e2cf None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.374031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1342c-13d5-45af-8b76-ea349001e2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.377015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2b336-022f-496b-8784-6b2826fcb2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:08.378372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2b336-022f-496b-8784-6b2826fcb2b4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.379393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2b336-022f-496b-8784-6b2826fcb2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.380939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2b336-022f-496b-8784-6b2826fcb2b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.396116 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2b336-022f-496b-8784-6b2826fcb2b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:08.398116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-90e1342c-13d5-45af-8b76-ea349001e2cf None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:08.399496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-90e1342c-13d5-45af-8b76-ea349001e2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-78e2b336-022f-496b-8784-6b2826fcb2b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:08.405447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:08.406256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:08.407483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 32/67] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:08.408558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2b336-022f-496b-8784-6b2826fcb2b4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:08.426929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:08.492509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 36/68] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:08.493793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1342c-13d5-45af-8b76-ea349001e2cf None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:08.513438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:08.524948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2b336-022f-496b-8784-6b2826fcb2b4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:08.561516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e39f1-8f16-4319-9aba-ca3a2ee496d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:08.562557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e39f1-8f16-4319-9aba-ca3a2ee496d9 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.563048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e39f1-8f16-4319-9aba-ca3a2ee496d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.563473 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e39f1-8f16-4319-9aba-ca3a2ee496d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.576862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e39f1-8f16-4319-9aba-ca3a2ee496d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-b14e39f1-8f16-4319-9aba-ca3a2ee496d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:08.585180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:08.586049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 33/69] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:08.587233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e39f1-8f16-4319-9aba-ca3a2ee496d9 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:08.610573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1342c-13d5-45af-8b76-ea349001e2cf None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:08.647206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57e50c-7ebb-4522-933c-a57635855629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:08.648142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57e50c-7ebb-4522-933c-a57635855629 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.648432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57e50c-7ebb-4522-933c-a57635855629 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.648751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57e50c-7ebb-4522-933c-a57635855629 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.649965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57e50c-7ebb-4522-933c-a57635855629 None admin] RBAC: Adding query filter params (name=member) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:08.650298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57e50c-7ebb-4522-933c-a57635855629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:08.651709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57e50c-7ebb-4522-933c-a57635855629 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:08.660850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 37/70] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:08.662126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8c585-dcfa-4d65-829e-a31f25627403 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.662427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8c585-dcfa-4d65-829e-a31f25627403 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.662714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a8c585-dcfa-4d65-829e-a31f25627403 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.663957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 38/71] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:08.664835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8c585-dcfa-4d65-829e-a31f25627403 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:08.677093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:08.708327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e39f1-8f16-4319-9aba-ca3a2ee496d9 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:08.742066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437cc47-377f-462a-a305-0f07ce9a895d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:08.742945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437cc47-377f-462a-a305-0f07ce9a895d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.743185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437cc47-377f-462a-a305-0f07ce9a895d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.743470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437cc47-377f-462a-a305-0f07ce9a895d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.747513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:08.754207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437cc47-377f-462a-a305-0f07ce9a895d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-2437cc47-377f-462a-a305-0f07ce9a895d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:08.762285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:08.763126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:08.763858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 34/72] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:08.764466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437cc47-377f-462a-a305-0f07ce9a895d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:08.774380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a8c585-dcfa-4d65-829e-a31f25627403 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:08.814894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae7731-036f-44d2-9fcb-3db70eacf10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:08.815578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7731-036f-44d2-9fcb-3db70eacf10c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.815868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7731-036f-44d2-9fcb-3db70eacf10c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.816150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae7731-036f-44d2-9fcb-3db70eacf10c None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.816958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae7731-036f-44d2-9fcb-3db70eacf10c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:08.817278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae7731-036f-44d2-9fcb-3db70eacf10c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:08.818698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae7731-036f-44d2-9fcb-3db70eacf10c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:08.828192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 39/73] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:08.829492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f4545-05f5-471d-9244-29cfc8235658 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.829866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f4545-05f5-471d-9244-29cfc8235658 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.830184 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f4545-05f5-471d-9244-29cfc8235658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.850901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c96f4545-05f5-471d-9244-29cfc8235658 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:08.852132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c96f4545-05f5-471d-9244-29cfc8235658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:08.876900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437cc47-377f-462a-a305-0f07ce9a895d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:08.906852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:08.913247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77022a-9ef4-4c7f-8360-13451c12e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:08.914109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77022a-9ef4-4c7f-8360-13451c12e725 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.914400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77022a-9ef4-4c7f-8360-13451c12e725 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.914678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77022a-9ef4-4c7f-8360-13451c12e725 None admin] PATH_INFO: `/v3/users/572c94b6640b44cc8fb0508eda65e95d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.920098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 40/74] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:08.922728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246385b-abbb-4b45-9de3-d8837a320dbb None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:08.922728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246385b-abbb-4b45-9de3-d8837a320dbb None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:08.922728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246385b-abbb-4b45-9de3-d8837a320dbb None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:08.923265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 41/75] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:08.924047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246385b-abbb-4b45-9de3-d8837a320dbb None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:08.928606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77022a-9ef4-4c7f-8360-13451c12e725 None admin] RBAC: Authorizing `identity:get_user(user_id=572c94b6640b44cc8fb0508eda65e95d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:08.930080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77022a-9ef4-4c7f-8360-13451c12e725 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:08.940531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 35/76] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:08 2022] GET /identity/v3/users/572c94b6640b44cc8fb0508eda65e95d => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:08.940531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77022a-9ef4-4c7f-8360-13451c12e725 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.003502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:09.035594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246385b-abbb-4b45-9de3-d8837a320dbb None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.039044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77022a-9ef4-4c7f-8360-13451c12e725 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.070625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508585e3-fffe-4c71-b10c-4634eefaa465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.071304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508585e3-fffe-4c71-b10c-4634eefaa465 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.071589 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508585e3-fffe-4c71-b10c-4634eefaa465 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.071870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508585e3-fffe-4c71-b10c-4634eefaa465 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.072651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508585e3-fffe-4c71-b10c-4634eefaa465 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:09.072966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508585e3-fffe-4c71-b10c-4634eefaa465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:09.073151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf89fbd-a962-4095-b5e3-c11cc53ab4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.074014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf89fbd-a962-4095-b5e3-c11cc53ab4a9 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.074301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf89fbd-a962-4095-b5e3-c11cc53ab4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.074395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508585e3-fffe-4c71-b10c-4634eefaa465 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:09.074579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf89fbd-a962-4095-b5e3-c11cc53ab4a9 None admin] PATH_INFO: `/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.075760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf89fbd-a962-4095-b5e3-c11cc53ab4a9 None admin] RBAC: Authorizing `identity:get_user(user_id=19e6b8d9d8f94b0c8f316b8b2060e76d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:09.077231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf89fbd-a962-4095-b5e3-c11cc53ab4a9 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:09.078836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 36/77] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:08 2022] GET /identity/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:09.079988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f0486-27d8-4e27-a698-8e37d14a1fc6 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.080361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f0486-27d8-4e27-a698-8e37d14a1fc6 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.080672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f0486-27d8-4e27-a698-8e37d14a1fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.084635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 42/78] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:09.085632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508585e3-fffe-4c71-b10c-4634eefaa465 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.096528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c47f0486-27d8-4e27-a698-8e37d14a1fc6 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:09.097750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c47f0486-27d8-4e27-a698-8e37d14a1fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:09.160004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:09.174438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 37/79] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:09.175481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f0486-27d8-4e27-a698-8e37d14a1fc6 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.207232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508585e3-fffe-4c71-b10c-4634eefaa465 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.236710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:09.242070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a99ae9-34b8-459e-89f3-8ecf725443a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.242721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a99ae9-34b8-459e-89f3-8ecf725443a5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.243006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a99ae9-34b8-459e-89f3-8ecf725443a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.243282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a99ae9-34b8-459e-89f3-8ecf725443a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.276171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f0486-27d8-4e27-a698-8e37d14a1fc6 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.309101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a99ae9-34b8-459e-89f3-8ecf725443a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:09.309936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf3bf2-cf28-4a62-bbc3-1365a082ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.310579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a99ae9-34b8-459e-89f3-8ecf725443a5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:09.310729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf3bf2-cf28-4a62-bbc3-1365a082ddec None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.311019 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf3bf2-cf28-4a62-bbc3-1365a082ddec None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.311308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf3bf2-cf28-4a62-bbc3-1365a082ddec None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.312480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf3bf2-cf28-4a62-bbc3-1365a082ddec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:09.313943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf3bf2-cf28-4a62-bbc3-1365a082ddec None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:09.320937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 38/80] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:09.320937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf3bf2-cf28-4a62-bbc3-1365a082ddec None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.372743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:09.410527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 43/81] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:09.416574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a99ae9-34b8-459e-89f3-8ecf725443a5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.416686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf3bf2-cf28-4a62-bbc3-1365a082ddec None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.446598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:09.451641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15416014-369c-4e1e-a6d7-727fdd7672ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.452299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416014-369c-4e1e-a6d7-727fdd7672ea None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.452611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416014-369c-4e1e-a6d7-727fdd7672ea None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.452907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416014-369c-4e1e-a6d7-727fdd7672ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.514118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a99ae9-34b8-459e-89f3-8ecf725443a5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.519760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15416014-369c-4e1e-a6d7-727fdd7672ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:09.521275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15416014-369c-4e1e-a6d7-727fdd7672ea None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:09.548249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdee87-0a65-4dc6-9157-be310c598641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.549619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdee87-0a65-4dc6-9157-be310c598641 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.549904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdee87-0a65-4dc6-9157-be310c598641 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.550193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdee87-0a65-4dc6-9157-be310c598641 None admin] PATH_INFO: `/v3/users/572c94b6640b44cc8fb0508eda65e95d` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.551394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdee87-0a65-4dc6-9157-be310c598641 None admin] RBAC: Authorizing `identity:get_user(user_id=572c94b6640b44cc8fb0508eda65e95d)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:09.552856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdee87-0a65-4dc6-9157-be310c598641 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:09.554478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 44/82] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:09 2022] GET /identity/v3/users/572c94b6640b44cc8fb0508eda65e95d => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:09.555243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdee87-0a65-4dc6-9157-be310c598641 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.575173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:09.621469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 39/83] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:09.622283 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15416014-369c-4e1e-a6d7-727fdd7672ea None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.652691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:09.657954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdee87-0a65-4dc6-9157-be310c598641 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.692077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27aec3-3770-42e8-b62f-93eec077e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.692912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27aec3-3770-42e8-b62f-93eec077e5b6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.693200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27aec3-3770-42e8-b62f-93eec077e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.693482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27aec3-3770-42e8-b62f-93eec077e5b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.703632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27aec3-3770-42e8-b62f-93eec077e5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-5b27aec3-3770-42e8-b62f-93eec077e5b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:09.712332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:09.713227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:09.713966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 45/84] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:09.714578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27aec3-3770-42e8-b62f-93eec077e5b6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.724065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15416014-369c-4e1e-a6d7-727fdd7672ea None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.758991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4128e5-1f3c-47fa-8aac-509d55926105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.759697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4128e5-1f3c-47fa-8aac-509d55926105 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.759981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4128e5-1f3c-47fa-8aac-509d55926105 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.760259 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4128e5-1f3c-47fa-8aac-509d55926105 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.761016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4128e5-1f3c-47fa-8aac-509d55926105 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:09.762475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4128e5-1f3c-47fa-8aac-509d55926105 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:09.788552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 40/85] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:35:09 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:35:09.789405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4128e5-1f3c-47fa-8aac-509d55926105 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.806613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:09.827047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27aec3-3770-42e8-b62f-93eec077e5b6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.866900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e45af-9fed-4c28-8d0c-70b72e0034b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.867798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e45af-9fed-4c28-8d0c-70b72e0034b1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.868231 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e45af-9fed-4c28-8d0c-70b72e0034b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.868519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e45af-9fed-4c28-8d0c-70b72e0034b1 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:09.869873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e45af-9fed-4c28-8d0c-70b72e0034b1 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:09.871508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e45af-9fed-4c28-8d0c-70b72e0034b1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:09.873536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 46/86] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:09 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:09.874212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e45af-9fed-4c28-8d0c-70b72e0034b1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:09.910733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:09.919241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4128e5-1f3c-47fa-8aac-509d55926105 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:09.965688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d1f9d-1a73-4efe-a7b0-677e1f88ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:09.966500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d1f9d-1a73-4efe-a7b0-677e1f88ed7c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:09.966662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d1f9d-1a73-4efe-a7b0-677e1f88ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:09.966950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d1f9d-1a73-4efe-a7b0-677e1f88ed7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:10.006514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e45af-9fed-4c28-8d0c-70b72e0034b1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:10.055120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d1f9d-1a73-4efe-a7b0-677e1f88ed7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:10.056702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d1f9d-1a73-4efe-a7b0-677e1f88ed7c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:10.058811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95997091-d145-46e3-9311-cd1f41393971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:10.059835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95997091-d145-46e3-9311-cd1f41393971 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:10.060118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95997091-d145-46e3-9311-cd1f41393971 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:10.060392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95997091-d145-46e3-9311-cd1f41393971 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:10.061714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95997091-d145-46e3-9311-cd1f41393971 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:10.063181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95997091-d145-46e3-9311-cd1f41393971 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:10.065988 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 47/87] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:09 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:10.066622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95997091-d145-46e3-9311-cd1f41393971 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:10.073715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:10.172762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:10.179644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95997091-d145-46e3-9311-cd1f41393971 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:10.194616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 41/88] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:10.195631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d1f9d-1a73-4efe-a7b0-677e1f88ed7c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:10.226235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45614a-c2e7-42b2-8ad8-218a1cdca136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:10.227489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45614a-c2e7-42b2-8ad8-218a1cdca136 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:10.227772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45614a-c2e7-42b2-8ad8-218a1cdca136 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:10.228054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45614a-c2e7-42b2-8ad8-218a1cdca136 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:10.229137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45614a-c2e7-42b2-8ad8-218a1cdca136 None admin] RBAC: Adding query filter params (name=member) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:10.229452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45614a-c2e7-42b2-8ad8-218a1cdca136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:10.231532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45614a-c2e7-42b2-8ad8-218a1cdca136 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:10.244105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 48/89] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:10.245083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45614a-c2e7-42b2-8ad8-218a1cdca136 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:10.326457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d1f9d-1a73-4efe-a7b0-677e1f88ed7c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:10.344827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:10.365675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45614a-c2e7-42b2-8ad8-218a1cdca136 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:10.379019 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40781c-6420-4a70-8d45-5245ca12df93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:10.379916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40781c-6420-4a70-8d45-5245ca12df93 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:10.380635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40781c-6420-4a70-8d45-5245ca12df93 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:10.381060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40781c-6420-4a70-8d45-5245ca12df93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:10.382751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40781c-6420-4a70-8d45-5245ca12df93 None admin] RBAC: Adding query filter params (role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8, user.id=572c94b6640b44cc8fb0508eda65e95d) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:10.383316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40781c-6420-4a70-8d45-5245ca12df93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:10.385589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40781c-6420-4a70-8d45-5245ca12df93 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:10.394194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 42/90] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:10 2022] GET /identity/v3/role_assignments?user.id=572c94b6640b44cc8fb0508eda65e95d&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=86c49ec8587244e48fab81862ddb3641 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:10.395267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40781c-6420-4a70-8d45-5245ca12df93 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:10.425115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fd151-bfff-48b0-afcb-b97a6acd4515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:10.425920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fd151-bfff-48b0-afcb-b97a6acd4515 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:10.426218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fd151-bfff-48b0-afcb-b97a6acd4515 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:10.426505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fd151-bfff-48b0-afcb-b97a6acd4515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:10.427469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fd151-bfff-48b0-afcb-b97a6acd4515 None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8, user.id=19e6b8d9d8f94b0c8f316b8b2060e76d) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:10.428038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fd151-bfff-48b0-afcb-b97a6acd4515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:10.429452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fd151-bfff-48b0-afcb-b97a6acd4515 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:10.448461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 49/91] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:10 2022] GET /identity/v3/role_assignments?user.id=19e6b8d9d8f94b0c8f316b8b2060e76d&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:10.449696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fd151-bfff-48b0-afcb-b97a6acd4515 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:10.503992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:10.590041 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40781c-6420-4a70-8d45-5245ca12df93 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:10.597394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fd151-bfff-48b0-afcb-b97a6acd4515 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:10.637642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8445cb-c41d-471c-96e5-7e8a809a5d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:10.639215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8445cb-c41d-471c-96e5-7e8a809a5d8f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:10.639682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8445cb-c41d-471c-96e5-7e8a809a5d8f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:10.640107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8445cb-c41d-471c-96e5-7e8a809a5d8f None admin] PATH_INFO: `/v3/users/572c94b6640b44cc8fb0508eda65e95d` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:10.641807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8445cb-c41d-471c-96e5-7e8a809a5d8f None admin] RBAC: Authorizing `identity:get_user(user_id=572c94b6640b44cc8fb0508eda65e95d)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:10.644412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8445cb-c41d-471c-96e5-7e8a809a5d8f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:10.645459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8637e9e-d2d6-4501-bc6a-8623a8b60e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:10.646422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8637e9e-d2d6-4501-bc6a-8623a8b60e33 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:10.646658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 50/92] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:10 2022] GET /identity/v3/users/572c94b6640b44cc8fb0508eda65e95d => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:10.651988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8637e9e-d2d6-4501-bc6a-8623a8b60e33 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:10.651988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8637e9e-d2d6-4501-bc6a-8623a8b60e33 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:10.653275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8637e9e-d2d6-4501-bc6a-8623a8b60e33 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:10.654409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8445cb-c41d-471c-96e5-7e8a809a5d8f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:10.656143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8637e9e-d2d6-4501-bc6a-8623a8b60e33 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:10.662278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 43/93] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:10 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:10.668503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8637e9e-d2d6-4501-bc6a-8623a8b60e33 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:10.739681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:10.815909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:10.832641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8445cb-c41d-471c-96e5-7e8a809a5d8f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:10.847380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8637e9e-d2d6-4501-bc6a-8623a8b60e33 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:10.896047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12ba72-2d57-4bc6-98e5-bf7703967b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:10.898009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12ba72-2d57-4bc6-98e5-bf7703967b7c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:10.898301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12ba72-2d57-4bc6-98e5-bf7703967b7c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:10.898581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa12ba72-2d57-4bc6-98e5-bf7703967b7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:10.913859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be372-502c-4706-9d31-ea54141c393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:10.914541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be372-502c-4706-9d31-ea54141c393d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:10.914828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be372-502c-4706-9d31-ea54141c393d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:10.915111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be372-502c-4706-9d31-ea54141c393d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:10.915955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657be372-502c-4706-9d31-ea54141c393d None admin] RBAC: Adding query filter params (role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8, user.id=572c94b6640b44cc8fb0508eda65e95d) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:10.916277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657be372-502c-4706-9d31-ea54141c393d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:10.917777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657be372-502c-4706-9d31-ea54141c393d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:10.930486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 44/94] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:10 2022] GET /identity/v3/role_assignments?user.id=572c94b6640b44cc8fb0508eda65e95d&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:11.014871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12ba72-2d57-4bc6-98e5-bf7703967b7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:11.016380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa12ba72-2d57-4bc6-98e5-bf7703967b7c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:11.149181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:11.162247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 51/95] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:11.167669 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be372-502c-4706-9d31-ea54141c393d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:11.301421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be372-502c-4706-9d31-ea54141c393d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:11.316937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:11.350622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d52911-05b7-4762-a599-6b0816027fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:11.351502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d52911-05b7-4762-a599-6b0816027fb4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:11.351792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d52911-05b7-4762-a599-6b0816027fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:11.352080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d52911-05b7-4762-a599-6b0816027fb4 None admin] PATH_INFO: `/v3/projects/1b89af06fd92444589c10d61e5cbc1aa` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:11.357317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d52911-05b7-4762-a599-6b0816027fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=1b89af06fd92444589c10d61e5cbc1aa)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:11.358785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d52911-05b7-4762-a599-6b0816027fb4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:11.360832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 45/96] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:11 2022] GET /identity/v3/projects/1b89af06fd92444589c10d61e5cbc1aa => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:11.366862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12ba72-2d57-4bc6-98e5-bf7703967b7c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:11.481309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa12ba72-2d57-4bc6-98e5-bf7703967b7c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:11.525034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38e585-42b9-4c7c-a540-df39e12c4eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:11.526057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38e585-42b9-4c7c-a540-df39e12c4eaf None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:11.526540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38e585-42b9-4c7c-a540-df39e12c4eaf None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:11.527015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38e585-42b9-4c7c-a540-df39e12c4eaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:11.528050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38e585-42b9-4c7c-a540-df39e12c4eaf None admin] RBAC: Adding query filter params (role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=1b89af06fd92444589c10d61e5cbc1aa, user.id=572c94b6640b44cc8fb0508eda65e95d) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:11.528606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38e585-42b9-4c7c-a540-df39e12c4eaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:11.530259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38e585-42b9-4c7c-a540-df39e12c4eaf None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:11.537570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 52/97] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:11 2022] GET /identity/v3/role_assignments?user.id=572c94b6640b44cc8fb0508eda65e95d&scope.project.id=1b89af06fd92444589c10d61e5cbc1aa&role.id=86c49ec8587244e48fab81862ddb3641 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:12.413978 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350ea6d-4f40-4e4f-982e-db48b9e3db9f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:12.414510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350ea6d-4f40-4e4f-982e-db48b9e3db9f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:12.414585 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350ea6d-4f40-4e4f-982e-db48b9e3db9f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:12.416178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 46/98] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:12.430119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31962081-88df-4710-9cde-c79e2f5bcebc None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:12.431006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31962081-88df-4710-9cde-c79e2f5bcebc None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:12.431507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31962081-88df-4710-9cde-c79e2f5bcebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:12.449975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-31962081-88df-4710-9cde-c79e2f5bcebc None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:12.451472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-31962081-88df-4710-9cde-c79e2f5bcebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:12.518899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:12.537849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 53/99] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:12.593446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350ea6d-4f40-4e4f-982e-db48b9e3db9f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:12.728214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350ea6d-4f40-4e4f-982e-db48b9e3db9f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:12.774708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b69c07-50d9-454e-b2d6-662cbad64353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:12.775519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b69c07-50d9-454e-b2d6-662cbad64353 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:12.775805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b69c07-50d9-454e-b2d6-662cbad64353 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:12.776085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b69c07-50d9-454e-b2d6-662cbad64353 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:12.789743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b69c07-50d9-454e-b2d6-662cbad64353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-94b69c07-50d9-454e-b2d6-662cbad64353 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:12.797480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:12.798294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:12.800091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 47/100] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:12.805213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31962081-88df-4710-9cde-c79e2f5bcebc None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:12.937444 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31962081-88df-4710-9cde-c79e2f5bcebc None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:12.960067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd0e60-f4a2-4907-8584-6d48dd48d5e4 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:12.960394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd0e60-f4a2-4907-8584-6d48dd48d5e4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:12.960661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd0e60-f4a2-4907-8584-6d48dd48d5e4 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:12.962062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 48/101] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:12.969946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c498c-b648-4096-8256-eefccb6f42a6 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:12.970249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c498c-b648-4096-8256-eefccb6f42a6 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:12.971131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c498c-b648-4096-8256-eefccb6f42a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:12.990342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d9d89-615d-45a8-8adf-bd8937b36e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:12.990971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d9d89-615d-45a8-8adf-bd8937b36e98 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:12.991261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d9d89-615d-45a8-8adf-bd8937b36e98 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:12.991543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d9d89-615d-45a8-8adf-bd8937b36e98 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:12.992311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d9d89-615d-45a8-8adf-bd8937b36e98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:12.992653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d9d89-615d-45a8-8adf-bd8937b36e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:12.994335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d9d89-615d-45a8-8adf-bd8937b36e98 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:12.998428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3d7c498c-b648-4096-8256-eefccb6f42a6 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:12.999630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3d7c498c-b648-4096-8256-eefccb6f42a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:13.017084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 54/102] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:13.023830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d9d89-615d-45a8-8adf-bd8937b36e98 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:13.066205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:13.100949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:13.148704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 49/103] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:13.191967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d9d89-615d-45a8-8adf-bd8937b36e98 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:13.206969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c498c-b648-4096-8256-eefccb6f42a6 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:13.235399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93e690-2e41-4feb-a37d-71eb4f6518e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:13.236966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93e690-2e41-4feb-a37d-71eb4f6518e1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:13.237275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93e690-2e41-4feb-a37d-71eb4f6518e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:13.237543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93e690-2e41-4feb-a37d-71eb4f6518e1 None admin] PATH_INFO: `/v3/users/b598f40388594a62814f9f05eba8aad3` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:13.265646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93e690-2e41-4feb-a37d-71eb4f6518e1 None admin] RBAC: Authorizing `identity:get_user(user_id=b598f40388594a62814f9f05eba8aad3)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:13.267052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93e690-2e41-4feb-a37d-71eb4f6518e1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:13.269248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 55/104] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:13 2022] GET /identity/v3/users/b598f40388594a62814f9f05eba8aad3 => generated 289 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:13.285920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93e690-2e41-4feb-a37d-71eb4f6518e1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:13.321021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c498c-b648-4096-8256-eefccb6f42a6 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:13.357505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2315b00-85e6-443b-af89-942a578eebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:13.358364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2315b00-85e6-443b-af89-942a578eebab None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:13.358657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2315b00-85e6-443b-af89-942a578eebab None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:13.358943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2315b00-85e6-443b-af89-942a578eebab None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:13.360781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2315b00-85e6-443b-af89-942a578eebab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:13.363951 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2315b00-85e6-443b-af89-942a578eebab None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:13.365947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 50/105] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:13.371781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2315b00-85e6-443b-af89-942a578eebab None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:13.409396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:13.481983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2315b00-85e6-443b-af89-942a578eebab None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:13.521195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17509c72-4c76-44cd-a31b-a6b5f532c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:13.521387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93e690-2e41-4feb-a37d-71eb4f6518e1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:13.521797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17509c72-4c76-44cd-a31b-a6b5f532c32a None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:13.522070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17509c72-4c76-44cd-a31b-a6b5f532c32a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:13.522360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17509c72-4c76-44cd-a31b-a6b5f532c32a None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:13.523341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17509c72-4c76-44cd-a31b-a6b5f532c32a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:13.524846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17509c72-4c76-44cd-a31b-a6b5f532c32a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:13.529592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:13.554120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 51/106] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:35:13 2022] POST /identity/v3/projects => generated 302 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:35:13.586879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837e50f-dd86-4fe3-915f-aad3e6e4d728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:13.587622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837e50f-dd86-4fe3-915f-aad3e6e4d728 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:13.587918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837e50f-dd86-4fe3-915f-aad3e6e4d728 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:13.588205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837e50f-dd86-4fe3-915f-aad3e6e4d728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:13.688301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837e50f-dd86-4fe3-915f-aad3e6e4d728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:13.696321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837e50f-dd86-4fe3-915f-aad3e6e4d728 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:13.746810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f41874-6637-464a-a04f-f50c46f05f03 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:13.747111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f41874-6637-464a-a04f-f50c46f05f03 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:13.747612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f41874-6637-464a-a04f-f50c46f05f03 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:13.749709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 52/107] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:13.758181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97f7b4-4417-4d73-b5da-2bc2d6112f9d None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:13.758811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97f7b4-4417-4d73-b5da-2bc2d6112f9d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:13.759316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97f7b4-4417-4d73-b5da-2bc2d6112f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:13.791181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0e97f7b4-4417-4d73-b5da-2bc2d6112f9d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:13.795022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0e97f7b4-4417-4d73-b5da-2bc2d6112f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:13.855209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 56/108] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 23:35:13.856937 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a94f-65b7-44c0-be5b-1d50b8cdc3e5 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:13.857449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a94f-65b7-44c0-be5b-1d50b8cdc3e5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:13.862762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a94f-65b7-44c0-be5b-1d50b8cdc3e5 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:13.864514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 57/109] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:13.865544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a94f-65b7-44c0-be5b-1d50b8cdc3e5 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:13.869798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:13.889422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 53/110] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:13.890390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8d84d-603f-4c69-b496-398f703a4a0c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:13.890686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8d84d-603f-4c69-b496-398f703a4a0c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:13.890970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8d84d-603f-4c69-b496-398f703a4a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:13.925659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-22c8d84d-603f-4c69-b496-398f703a4a0c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:13.926900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-22c8d84d-603f-4c69-b496-398f703a4a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:13.938977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:13.990772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a94f-65b7-44c0-be5b-1d50b8cdc3e5 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:14.009627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 54/111] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:14.010491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8d84d-603f-4c69-b496-398f703a4a0c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:14.032489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8af84-2345-44f2-ac8a-b3be294111a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:14.033922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8af84-2345-44f2-ac8a-b3be294111a5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.034211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8af84-2345-44f2-ac8a-b3be294111a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.034491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8af84-2345-44f2-ac8a-b3be294111a5 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.035834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8af84-2345-44f2-ac8a-b3be294111a5 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:14.037683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8af84-2345-44f2-ac8a-b3be294111a5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:14.039519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 58/112] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:13 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:14.046069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8af84-2345-44f2-ac8a-b3be294111a5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:14.124463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:14.156906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8af84-2345-44f2-ac8a-b3be294111a5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:14.162127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8d84d-603f-4c69-b496-398f703a4a0c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:14.203530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ea56b-ccb1-4c15-be6e-2c5cb493ade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:14.205447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ea56b-ccb1-4c15-be6e-2c5cb493ade4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.205940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ea56b-ccb1-4c15-be6e-2c5cb493ade4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.206425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ea56b-ccb1-4c15-be6e-2c5cb493ade4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.220111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ea56b-ccb1-4c15-be6e-2c5cb493ade4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:14.227385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72559bf-2f23-4c69-bb00-3eb9037f49a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:14.228052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72559bf-2f23-4c69-bb00-3eb9037f49a7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.228336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72559bf-2f23-4c69-bb00-3eb9037f49a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.228643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72559bf-2f23-4c69-bb00-3eb9037f49a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.229492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72559bf-2f23-4c69-bb00-3eb9037f49a7 None admin] RBAC: Adding query filter params (role.id=3f15101d95de479cacc1f6d91a2ff504, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8, user.id=b598f40388594a62814f9f05eba8aad3) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:14.229807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72559bf-2f23-4c69-bb00-3eb9037f49a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-f15ea56b-ccb1-4c15-be6e-2c5cb493ade4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:14.230870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:14.231697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:14.232464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:14.233080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72559bf-2f23-4c69-bb00-3eb9037f49a7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:14.235014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 55/113] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:14.236616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ea56b-ccb1-4c15-be6e-2c5cb493ade4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:14.243284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 59/114] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:14 2022] GET /identity/v3/role_assignments?user.id=b598f40388594a62814f9f05eba8aad3&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=3f15101d95de479cacc1f6d91a2ff504 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:14.244626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908859d-83ef-4ded-8f9c-2e8f7bca28ba None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.244910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908859d-83ef-4ded-8f9c-2e8f7bca28ba None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.245190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908859d-83ef-4ded-8f9c-2e8f7bca28ba None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.246445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 60/115] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:14.247160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908859d-83ef-4ded-8f9c-2e8f7bca28ba None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:14.253225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:14.373468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ea56b-ccb1-4c15-be6e-2c5cb493ade4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:14.401813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908859d-83ef-4ded-8f9c-2e8f7bca28ba None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:14.414929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3feccf-d610-408c-93fe-4b3c2ab6eda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:14.415750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3feccf-d610-408c-93fe-4b3c2ab6eda3 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.416130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3feccf-d610-408c-93fe-4b3c2ab6eda3 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.416420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3feccf-d610-408c-93fe-4b3c2ab6eda3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.428929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3feccf-d610-408c-93fe-4b3c2ab6eda3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-2a3feccf-d610-408c-93fe-4b3c2ab6eda3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:14.437146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:14.437975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:14.438706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:14.440171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a921cd-1716-41c4-b19f-48769df3a181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:14.440171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a921cd-1716-41c4-b19f-48769df3a181 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.440171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a921cd-1716-41c4-b19f-48769df3a181 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.440171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a921cd-1716-41c4-b19f-48769df3a181 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.441305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a921cd-1716-41c4-b19f-48769df3a181 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:14.441305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a921cd-1716-41c4-b19f-48769df3a181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:14.448601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 56/116] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:14.448843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a921cd-1716-41c4-b19f-48769df3a181 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:14.448987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c2f94-2894-4de3-b0fa-a96739a1c1a1 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.448987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c2f94-2894-4de3-b0fa-a96739a1c1a1 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.448987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c2f94-2894-4de3-b0fa-a96739a1c1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.449279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:14.451929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 61/117] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:14.452826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a921cd-1716-41c4-b19f-48769df3a181 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:14.468438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-5e3c2f94-2894-4de3-b0fa-a96739a1c1a1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:14.469870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5e3c2f94-2894-4de3-b0fa-a96739a1c1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:14.547854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:14.559197 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 57/118] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:14.560440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c2f94-2894-4de3-b0fa-a96739a1c1a1 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:14.561083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a921cd-1716-41c4-b19f-48769df3a181 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:14.596786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580fade-3c81-4670-806b-45ecb5768b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:14.597506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580fade-3c81-4670-806b-45ecb5768b6d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.597756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580fade-3c81-4670-806b-45ecb5768b6d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.598041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580fade-3c81-4670-806b-45ecb5768b6d None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.598808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580fade-3c81-4670-806b-45ecb5768b6d None admin] RBAC: Adding query filter params (name=member) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:14.599125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580fade-3c81-4670-806b-45ecb5768b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:14.600521 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580fade-3c81-4670-806b-45ecb5768b6d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:14.611463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 62/119] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:14.611463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580fade-3c81-4670-806b-45ecb5768b6d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:14.671876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:14.683048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c2f94-2894-4de3-b0fa-a96739a1c1a1 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:14.722000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44132bd1-1d57-46f3-9b95-6a89202f1f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:14.723335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44132bd1-1d57-46f3-9b95-6a89202f1f29 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.723916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44132bd1-1d57-46f3-9b95-6a89202f1f29 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.724398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44132bd1-1d57-46f3-9b95-6a89202f1f29 None admin] PATH_INFO: `/v3/users/572c94b6640b44cc8fb0508eda65e95d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.725919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44132bd1-1d57-46f3-9b95-6a89202f1f29 None admin] RBAC: Authorizing `identity:get_user(user_id=572c94b6640b44cc8fb0508eda65e95d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:14.727740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44132bd1-1d57-46f3-9b95-6a89202f1f29 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:14.729984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 58/120] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:14 2022] GET /identity/v3/users/572c94b6640b44cc8fb0508eda65e95d => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:14.731092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44132bd1-1d57-46f3-9b95-6a89202f1f29 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:14.737475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580fade-3c81-4670-806b-45ecb5768b6d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:14.818817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f18537-bf4b-484b-bf20-432f1fa87271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:14.821174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f18537-bf4b-484b-bf20-432f1fa87271 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:14.822833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f18537-bf4b-484b-bf20-432f1fa87271 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:14.823319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f18537-bf4b-484b-bf20-432f1fa87271 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:14.826515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:14.851133 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f18537-bf4b-484b-bf20-432f1fa87271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-95f18537-bf4b-484b-bf20-432f1fa87271 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:14.862028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:14.862895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:14.863641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:14.870992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 63/121] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:14.874603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f18537-bf4b-484b-bf20-432f1fa87271 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:14.915559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44132bd1-1d57-46f3-9b95-6a89202f1f29 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:15.000983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992370d-8da0-4ac1-8dc6-3f0c04a009d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:15.004149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992370d-8da0-4ac1-8dc6-3f0c04a009d3 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.004430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992370d-8da0-4ac1-8dc6-3f0c04a009d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.004732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992370d-8da0-4ac1-8dc6-3f0c04a009d3 None admin] PATH_INFO: `/v3/users/572c94b6640b44cc8fb0508eda65e95d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.006125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992370d-8da0-4ac1-8dc6-3f0c04a009d3 None admin] RBAC: Authorizing `identity:get_user(user_id=572c94b6640b44cc8fb0508eda65e95d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:15.007825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992370d-8da0-4ac1-8dc6-3f0c04a009d3 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:15.009800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 59/122] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:14 2022] GET /identity/v3/users/572c94b6640b44cc8fb0508eda65e95d => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:15.010808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992370d-8da0-4ac1-8dc6-3f0c04a009d3 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:15.085285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f18537-bf4b-484b-bf20-432f1fa87271 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:15.106390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:15.122884 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f34847-b925-494b-a619-c0ec9ff825e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:15.124095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f34847-b925-494b-a619-c0ec9ff825e1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.124613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f34847-b925-494b-a619-c0ec9ff825e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.125153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f34847-b925-494b-a619-c0ec9ff825e1 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.126865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f34847-b925-494b-a619-c0ec9ff825e1 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:15.129009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f34847-b925-494b-a619-c0ec9ff825e1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:15.131400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 64/123] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:14 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:15.133152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf1e59-4440-4d5a-a377-c7d82a21ba2d None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.133647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf1e59-4440-4d5a-a377-c7d82a21ba2d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.134145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf1e59-4440-4d5a-a377-c7d82a21ba2d None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.139557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992370d-8da0-4ac1-8dc6-3f0c04a009d3 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:15.139953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 65/124] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:15.141110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf1e59-4440-4d5a-a377-c7d82a21ba2d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:15.176251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:15.183387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1855c-1043-40c9-92d9-0dd175eb9b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:15.184070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1855c-1043-40c9-92d9-0dd175eb9b14 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.184357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1855c-1043-40c9-92d9-0dd175eb9b14 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.184660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1855c-1043-40c9-92d9-0dd175eb9b14 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.185431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1855c-1043-40c9-92d9-0dd175eb9b14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:15.185799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1855c-1043-40c9-92d9-0dd175eb9b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:15.187487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1855c-1043-40c9-92d9-0dd175eb9b14 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:15.201334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 60/125] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:15.202166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1855c-1043-40c9-92d9-0dd175eb9b14 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:15.284459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf1e59-4440-4d5a-a377-c7d82a21ba2d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:15.322389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9309b-18f2-4416-8874-db9eb4c94565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:15.323265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9309b-18f2-4416-8874-db9eb4c94565 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.323550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9309b-18f2-4416-8874-db9eb4c94565 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.323829 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9309b-18f2-4416-8874-db9eb4c94565 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.325080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9309b-18f2-4416-8874-db9eb4c94565 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:15.326541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9309b-18f2-4416-8874-db9eb4c94565 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:15.328344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 66/126] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:15 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:15.329498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3879ec4-c6f2-4e4e-91bf-2574eb7366e7 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.329822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3879ec4-c6f2-4e4e-91bf-2574eb7366e7 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.330700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3879ec4-c6f2-4e4e-91bf-2574eb7366e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.335665 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1855c-1043-40c9-92d9-0dd175eb9b14 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:15.349031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d3879ec4-c6f2-4e4e-91bf-2574eb7366e7 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:15.350330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d3879ec4-c6f2-4e4e-91bf-2574eb7366e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:15.381062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618633-ecd5-439d-b70c-af620ba6e99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:15.382836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618633-ecd5-439d-b70c-af620ba6e99f None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.383583 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618633-ecd5-439d-b70c-af620ba6e99f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.384149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618633-ecd5-439d-b70c-af620ba6e99f None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/users/572c94b6640b44cc8fb0508eda65e95d/roles/32433ed911104bb68970c0a00d0ddd49` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.386791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618633-ecd5-439d-b70c-af620ba6e99f None admin] RBAC: Authorizing `identity:create_grant(project_id=129c32efbe2e4cc19ffc72afb96752e8, user_id=572c94b6640b44cc8fb0508eda65e95d, role_id=32433ed911104bb68970c0a00d0ddd49)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:15.389503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618633-ecd5-439d-b70c-af620ba6e99f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:15.409051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:15.410971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 61/127] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:35:15 2022] PUT /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/users/572c94b6640b44cc8fb0508eda65e95d/roles/32433ed911104bb68970c0a00d0ddd49 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:15.412119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618633-ecd5-439d-b70c-af620ba6e99f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:15.451647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 67/128] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:15.456699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3879ec4-c6f2-4e4e-91bf-2574eb7366e7 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:15.510163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:15.560832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618633-ecd5-439d-b70c-af620ba6e99f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:15.609697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465407c-041c-444a-b321-c4d7531c10c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:15.610609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465407c-041c-444a-b321-c4d7531c10c7 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.611529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465407c-041c-444a-b321-c4d7531c10c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.611820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465407c-041c-444a-b321-c4d7531c10c7 None admin] PATH_INFO: `/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.615768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465407c-041c-444a-b321-c4d7531c10c7 None admin] RBAC: Authorizing `identity:get_user(user_id=19e6b8d9d8f94b0c8f316b8b2060e76d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:15.617445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465407c-041c-444a-b321-c4d7531c10c7 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:15.619611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 62/129] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:15 2022] GET /identity/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:15.620731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465407c-041c-444a-b321-c4d7531c10c7 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:15.695622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3879ec4-c6f2-4e4e-91bf-2574eb7366e7 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:15.746288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53149877-9cee-447e-944c-933bc13b8d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:15.747205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53149877-9cee-447e-944c-933bc13b8d37 None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.747536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53149877-9cee-447e-944c-933bc13b8d37 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.747767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53149877-9cee-447e-944c-933bc13b8d37 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/users/572c94b6640b44cc8fb0508eda65e95d/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.749925 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53149877-9cee-447e-944c-933bc13b8d37 None admin] RBAC: Authorizing `identity:create_grant(project_id=129c32efbe2e4cc19ffc72afb96752e8, user_id=572c94b6640b44cc8fb0508eda65e95d, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:15.751378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53149877-9cee-447e-944c-933bc13b8d37 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:15.780632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465407c-041c-444a-b321-c4d7531c10c7 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:15.782405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 68/130] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:35:15 2022] PUT /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/users/572c94b6640b44cc8fb0508eda65e95d/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:15.783468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53149877-9cee-447e-944c-933bc13b8d37 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:15.817597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:15.832290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69526d-abcc-4a09-afd3-0053384800b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:15.833547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69526d-abcc-4a09-afd3-0053384800b5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:15.834041 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69526d-abcc-4a09-afd3-0053384800b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:15.834520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69526d-abcc-4a09-afd3-0053384800b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:15.852476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69526d-abcc-4a09-afd3-0053384800b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-aa69526d-abcc-4a09-afd3-0053384800b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:15.862141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:15.863091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:15.863839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:15.866560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 63/131] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:15.873164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69526d-abcc-4a09-afd3-0053384800b5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:15.877733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:15.993181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69526d-abcc-4a09-afd3-0053384800b5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:16.030553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67883a57-21b9-4298-9be9-a49595179cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:16.031249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67883a57-21b9-4298-9be9-a49595179cf5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:16.031535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67883a57-21b9-4298-9be9-a49595179cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:16.031825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67883a57-21b9-4298-9be9-a49595179cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:16.032632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67883a57-21b9-4298-9be9-a49595179cf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:16.033006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67883a57-21b9-4298-9be9-a49595179cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:16.034438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67883a57-21b9-4298-9be9-a49595179cf5 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:16.042798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53149877-9cee-447e-944c-933bc13b8d37 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:16.043761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 64/132] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:16.049466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67883a57-21b9-4298-9be9-a49595179cf5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:16.100623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:16.134659 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d01881-0915-466c-a2dc-1a743ce5f1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:16.135549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d01881-0915-466c-a2dc-1a743ce5f1d7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:16.135839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d01881-0915-466c-a2dc-1a743ce5f1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:16.136119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d01881-0915-466c-a2dc-1a743ce5f1d7 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:16.141309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d01881-0915-466c-a2dc-1a743ce5f1d7 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:16.143014 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d01881-0915-466c-a2dc-1a743ce5f1d7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:16.145097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 69/133] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:15 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:16.150470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d01881-0915-466c-a2dc-1a743ce5f1d7 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:16.160227 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67883a57-21b9-4298-9be9-a49595179cf5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:16.195542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c024942-4333-4786-ac0f-3e7b7f8af96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:16.196426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c024942-4333-4786-ac0f-3e7b7f8af96b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:16.196767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c024942-4333-4786-ac0f-3e7b7f8af96b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:16.197054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c024942-4333-4786-ac0f-3e7b7f8af96b None admin] PATH_INFO: `/v3/users/572c94b6640b44cc8fb0508eda65e95d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:16.198292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c024942-4333-4786-ac0f-3e7b7f8af96b None admin] RBAC: Authorizing `identity:get_user(user_id=572c94b6640b44cc8fb0508eda65e95d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:16.199771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c024942-4333-4786-ac0f-3e7b7f8af96b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:16.201893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 65/134] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:16 2022] GET /identity/v3/users/572c94b6640b44cc8fb0508eda65e95d => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:16.207546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c024942-4333-4786-ac0f-3e7b7f8af96b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:16.315292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d01881-0915-466c-a2dc-1a743ce5f1d7 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:16.343978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:16.371643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e86cb6-e963-4166-bf00-592b6c3d7a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:16.373054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e86cb6-e963-4166-bf00-592b6c3d7a22 None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:16.373552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e86cb6-e963-4166-bf00-592b6c3d7a22 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:16.374134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e86cb6-e963-4166-bf00-592b6c3d7a22 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/users/19e6b8d9d8f94b0c8f316b8b2060e76d/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:16.376982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e86cb6-e963-4166-bf00-592b6c3d7a22 None admin] RBAC: Authorizing `identity:create_grant(project_id=129c32efbe2e4cc19ffc72afb96752e8, user_id=19e6b8d9d8f94b0c8f316b8b2060e76d, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:16.378353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c024942-4333-4786-ac0f-3e7b7f8af96b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:16.378878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e86cb6-e963-4166-bf00-592b6c3d7a22 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:16.393152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:16.395191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 70/135] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:35:16 2022] PUT /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/users/19e6b8d9d8f94b0c8f316b8b2060e76d/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:35:16.417646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511dbb4-0968-4232-a698-a57dbf2e6a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:16.418536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511dbb4-0968-4232-a698-a57dbf2e6a18 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:16.418826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511dbb4-0968-4232-a698-a57dbf2e6a18 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:16.419109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511dbb4-0968-4232-a698-a57dbf2e6a18 None admin] PATH_INFO: `/v3/projects/1b89af06fd92444589c10d61e5cbc1aa` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:16.420311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511dbb4-0968-4232-a698-a57dbf2e6a18 None admin] RBAC: Authorizing `identity:get_project(project_id=1b89af06fd92444589c10d61e5cbc1aa)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:16.421806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511dbb4-0968-4232-a698-a57dbf2e6a18 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:16.424639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 66/136] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:16 2022] GET /identity/v3/projects/1b89af06fd92444589c10d61e5cbc1aa => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:16.432727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e86cb6-e963-4166-bf00-592b6c3d7a22 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:16.549559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82623172-03f8-4c3d-9328-5a268b2e2788 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:16.550153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82623172-03f8-4c3d-9328-5a268b2e2788 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:16.550605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82623172-03f8-4c3d-9328-5a268b2e2788 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:16.552206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 67/137] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:16.557910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5d18f-2cf1-41dd-88e0-f563e217cc9c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:16.558449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5d18f-2cf1-41dd-88e0-f563e217cc9c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:16.558943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5d18f-2cf1-41dd-88e0-f563e217cc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:16.564605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e86cb6-e963-4166-bf00-592b6c3d7a22 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:16.596608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-eca5d18f-2cf1-41dd-88e0-f563e217cc9c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:16.602446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-eca5d18f-2cf1-41dd-88e0-f563e217cc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:16.609210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafca58-efc9-42bb-a9e2-9b90df0ccacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:16.610144 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafca58-efc9-42bb-a9e2-9b90df0ccacf None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:16.610431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafca58-efc9-42bb-a9e2-9b90df0ccacf None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:16.612557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafca58-efc9-42bb-a9e2-9b90df0ccacf None admin] PATH_INFO: `/v3/projects/1b89af06fd92444589c10d61e5cbc1aa/users/572c94b6640b44cc8fb0508eda65e95d/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:16.618511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafca58-efc9-42bb-a9e2-9b90df0ccacf None admin] RBAC: Authorizing `identity:create_grant(project_id=1b89af06fd92444589c10d61e5cbc1aa, user_id=572c94b6640b44cc8fb0508eda65e95d, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:16.619996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafca58-efc9-42bb-a9e2-9b90df0ccacf None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:16.653672 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 71/138] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:35:16 2022] PUT /identity/v3/projects/1b89af06fd92444589c10d61e5cbc1aa/users/572c94b6640b44cc8fb0508eda65e95d/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:16.723466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 68/139] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:16.772470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafca58-efc9-42bb-a9e2-9b90df0ccacf None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:16.811223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:16.901032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafca58-efc9-42bb-a9e2-9b90df0ccacf None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:16.960797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38e466-5662-48f7-8849-164fd09d0e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:16.961633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38e466-5662-48f7-8849-164fd09d0e8d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:16.961923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38e466-5662-48f7-8849-164fd09d0e8d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:16.962199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38e466-5662-48f7-8849-164fd09d0e8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:16.963601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38e466-5662-48f7-8849-164fd09d0e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:16.965101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38e466-5662-48f7-8849-164fd09d0e8d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:16.967080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 72/140] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:16.973269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5d18f-2cf1-41dd-88e0-f563e217cc9c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:17.095340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5d18f-2cf1-41dd-88e0-f563e217cc9c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:17.112515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:17.148278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf3b96-41ef-484a-bd2b-47c6d0e98d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:17.149463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf3b96-41ef-484a-bd2b-47c6d0e98d3b None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:17.149760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf3b96-41ef-484a-bd2b-47c6d0e98d3b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:17.150050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf3b96-41ef-484a-bd2b-47c6d0e98d3b None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:17.150796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf3b96-41ef-484a-bd2b-47c6d0e98d3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:17.152266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf3b96-41ef-484a-bd2b-47c6d0e98d3b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:17.186289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 69/141] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:35:16 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:35:17.289263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506351-ef14-4c1a-846b-6ac53ffd64ce None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:17.289589 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506351-ef14-4c1a-846b-6ac53ffd64ce None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:17.289750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506351-ef14-4c1a-846b-6ac53ffd64ce None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:17.291647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 73/142] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:17.299804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d57b98-ea45-4ef6-a355-96112b8e6a46 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:17.299804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d57b98-ea45-4ef6-a355-96112b8e6a46 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:17.299804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d57b98-ea45-4ef6-a355-96112b8e6a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:17.317363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c9d57b98-ea45-4ef6-a355-96112b8e6a46 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:17.318999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c9d57b98-ea45-4ef6-a355-96112b8e6a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:17.395136 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 70/143] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:17.418162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506351-ef14-4c1a-846b-6ac53ffd64ce None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:17.532471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb506351-ef14-4c1a-846b-6ac53ffd64ce None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:17.535173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:17.571324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e05a40-8c48-4edf-ab91-4405b100706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:17.572060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05a40-8c48-4edf-ab91-4405b100706f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:17.572352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05a40-8c48-4edf-ab91-4405b100706f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:17.573886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05a40-8c48-4edf-ab91-4405b100706f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:17.585085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e05a40-8c48-4edf-ab91-4405b100706f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-14e05a40-8c48-4edf-ab91-4405b100706f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:17.592962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:17.593775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:17.594509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 74/144] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:17.602077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d57b98-ea45-4ef6-a355-96112b8e6a46 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:17.723639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:17.729589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d57b98-ea45-4ef6-a355-96112b8e6a46 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:17.775309 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d823e-a7eb-4374-892a-73177c057e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:17.775995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d823e-a7eb-4374-892a-73177c057e2b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:17.776293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d823e-a7eb-4374-892a-73177c057e2b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:17.776594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d823e-a7eb-4374-892a-73177c057e2b None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:17.777365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d823e-a7eb-4374-892a-73177c057e2b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:17.777679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d823e-a7eb-4374-892a-73177c057e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:17.779093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d823e-a7eb-4374-892a-73177c057e2b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:17.791976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 71/145] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:17.797734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e05a40-8c48-4edf-ab91-4405b100706f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:17.945155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e05a40-8c48-4edf-ab91-4405b100706f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:17.992660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d89345-1193-4b84-bf1e-3f820989eb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:17.993544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d89345-1193-4b84-bf1e-3f820989eb66 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:17.993826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d89345-1193-4b84-bf1e-3f820989eb66 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:17.994099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d89345-1193-4b84-bf1e-3f820989eb66 None admin] PATH_INFO: `/v3/users/b598f40388594a62814f9f05eba8aad3` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:17.995376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d89345-1193-4b84-bf1e-3f820989eb66 None admin] RBAC: Authorizing `identity:get_user(user_id=b598f40388594a62814f9f05eba8aad3)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:17.996904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d89345-1193-4b84-bf1e-3f820989eb66 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:17.998543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 75/146] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:17 2022] GET /identity/v3/users/b598f40388594a62814f9f05eba8aad3 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:18.004564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d823e-a7eb-4374-892a-73177c057e2b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:18.131145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d823e-a7eb-4374-892a-73177c057e2b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:18.171797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c892d-e207-4675-a05c-280b0ef725b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:18.172700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c892d-e207-4675-a05c-280b0ef725b0 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:18.172986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c892d-e207-4675-a05c-280b0ef725b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:18.173794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c892d-e207-4675-a05c-280b0ef725b0 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:18.175006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c892d-e207-4675-a05c-280b0ef725b0 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:18.176544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c892d-e207-4675-a05c-280b0ef725b0 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:18.178332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 72/147] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:18 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:18.184053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d89345-1193-4b84-bf1e-3f820989eb66 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:18.218519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:18.300244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d89345-1193-4b84-bf1e-3f820989eb66 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:18.346347 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12749734-ea5d-4a8c-86a5-f288381ca145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:18.347307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749734-ea5d-4a8c-86a5-f288381ca145 None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:18.347591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749734-ea5d-4a8c-86a5-f288381ca145 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:18.347868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749734-ea5d-4a8c-86a5-f288381ca145 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/users/b598f40388594a62814f9f05eba8aad3/roles/3f15101d95de479cacc1f6d91a2ff504` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:18.349916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12749734-ea5d-4a8c-86a5-f288381ca145 None admin] RBAC: Authorizing `identity:create_grant(project_id=129c32efbe2e4cc19ffc72afb96752e8, user_id=b598f40388594a62814f9f05eba8aad3, role_id=3f15101d95de479cacc1f6d91a2ff504)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:18.351441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12749734-ea5d-4a8c-86a5-f288381ca145 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:18.377656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 76/148] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:35:18 2022] PUT /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/users/b598f40388594a62814f9f05eba8aad3/roles/3f15101d95de479cacc1f6d91a2ff504 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:18.821793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de53504-aae1-40f9-8bb9-78ecd9cf9757 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:18.822317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de53504-aae1-40f9-8bb9-78ecd9cf9757 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:18.822317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de53504-aae1-40f9-8bb9-78ecd9cf9757 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:18.824178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 73/149] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:18.839412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527cf9a-7927-407e-9179-1abc93980b70 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:18.839890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527cf9a-7927-407e-9179-1abc93980b70 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:18.839961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527cf9a-7927-407e-9179-1abc93980b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:18.858274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c527cf9a-7927-407e-9179-1abc93980b70 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:18.859508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c527cf9a-7927-407e-9179-1abc93980b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:18.944539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 77/150] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:18.994243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de53504-aae1-40f9-8bb9-78ecd9cf9757 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:19.029862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:19.117023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de53504-aae1-40f9-8bb9-78ecd9cf9757 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:19.154347 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2706e8-e895-46a4-b221-bfe28acf001c None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:19.154660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2706e8-e895-46a4-b221-bfe28acf001c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:19.154905 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2706e8-e895-46a4-b221-bfe28acf001c None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:19.156286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 78/151] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:19.174158 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107c4f-5d73-4a46-a349-5d9f05006a0c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:19.174465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107c4f-5d73-4a46-a349-5d9f05006a0c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:19.174719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06107c4f-5d73-4a46-a349-5d9f05006a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:19.190537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2239a59-dafe-4aa6-b10f-88a99792e5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:19.191355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2239a59-dafe-4aa6-b10f-88a99792e5e5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:19.191636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2239a59-dafe-4aa6-b10f-88a99792e5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:19.191911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2239a59-dafe-4aa6-b10f-88a99792e5e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:19.203501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-06107c4f-5d73-4a46-a349-5d9f05006a0c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:19.208947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-06107c4f-5d73-4a46-a349-5d9f05006a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:19.221511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2239a59-dafe-4aa6-b10f-88a99792e5e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-c2239a59-dafe-4aa6-b10f-88a99792e5e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:19.235105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:19.236874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:19.237612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 74/152] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:19.245165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:19.245519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2239a59-dafe-4aa6-b10f-88a99792e5e5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:19.329837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 79/153] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:19.375808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06107c4f-5d73-4a46-a349-5d9f05006a0c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:19.473337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:19.493017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2239a59-dafe-4aa6-b10f-88a99792e5e5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:19.531045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06107c4f-5d73-4a46-a349-5d9f05006a0c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:19.561856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163132-1cfa-48e5-8b1e-9fff11294091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:19.562560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163132-1cfa-48e5-8b1e-9fff11294091 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:19.562848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163132-1cfa-48e5-8b1e-9fff11294091 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:19.563133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1163132-1cfa-48e5-8b1e-9fff11294091 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:19.563931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1163132-1cfa-48e5-8b1e-9fff11294091 None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:19.564280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1163132-1cfa-48e5-8b1e-9fff11294091 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:19.565906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1163132-1cfa-48e5-8b1e-9fff11294091 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:19.578901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 75/154] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:19.580445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410b555-8d33-43fc-ad12-1896f7612ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:19.584926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163132-1cfa-48e5-8b1e-9fff11294091 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:19.585435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410b555-8d33-43fc-ad12-1896f7612ff8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:19.585738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410b555-8d33-43fc-ad12-1896f7612ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:19.586025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410b555-8d33-43fc-ad12-1896f7612ff8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:19.601461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410b555-8d33-43fc-ad12-1896f7612ff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-5410b555-8d33-43fc-ad12-1896f7612ff8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:19.612897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:19.613781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:19.614684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 80/155] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:19.621836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410b555-8d33-43fc-ad12-1896f7612ff8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:19.645512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:19.756475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1163132-1cfa-48e5-8b1e-9fff11294091 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:19.775928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410b555-8d33-43fc-ad12-1896f7612ff8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:19.808031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2adf3-cefc-4da8-87e5-be7069f5da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:19.808963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2adf3-cefc-4da8-87e5-be7069f5da31 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:19.809253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2adf3-cefc-4da8-87e5-be7069f5da31 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:19.809541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2adf3-cefc-4da8-87e5-be7069f5da31 None admin] PATH_INFO: `/v3/users/572c94b6640b44cc8fb0508eda65e95d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:19.810769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2adf3-cefc-4da8-87e5-be7069f5da31 None admin] RBAC: Authorizing `identity:get_user(user_id=572c94b6640b44cc8fb0508eda65e95d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:19.812235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c2adf3-cefc-4da8-87e5-be7069f5da31 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:19.814750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 76/156] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:19 2022] GET /identity/v3/users/572c94b6640b44cc8fb0508eda65e95d => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:19.820228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2adf3-cefc-4da8-87e5-be7069f5da31 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:19.831474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206a2bc-6da5-4cf0-be4a-fac599059e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:19.832230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206a2bc-6da5-4cf0-be4a-fac599059e70 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:19.832515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206a2bc-6da5-4cf0-be4a-fac599059e70 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:19.832813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206a2bc-6da5-4cf0-be4a-fac599059e70 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:19.833583 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206a2bc-6da5-4cf0-be4a-fac599059e70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:19.833904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206a2bc-6da5-4cf0-be4a-fac599059e70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:19.835408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206a2bc-6da5-4cf0-be4a-fac599059e70 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:19.848044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 81/157] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:19.853004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:19.853584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206a2bc-6da5-4cf0-be4a-fac599059e70 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:20.002179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:20.022269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2adf3-cefc-4da8-87e5-be7069f5da31 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:20.077681 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206a2bc-6da5-4cf0-be4a-fac599059e70 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:20.093984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b4388-ef5f-4536-9866-4e3815655ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:20.094930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b4388-ef5f-4536-9866-4e3815655ccb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:20.095972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b4388-ef5f-4536-9866-4e3815655ccb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:20.096439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b4388-ef5f-4536-9866-4e3815655ccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:20.152452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a415d-0c92-42d4-80e9-201bd2736d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:20.153715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a415d-0c92-42d4-80e9-201bd2736d16 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:20.154188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a415d-0c92-42d4-80e9-201bd2736d16 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:20.154675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a415d-0c92-42d4-80e9-201bd2736d16 None admin] PATH_INFO: `/v3/users/572c94b6640b44cc8fb0508eda65e95d` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:20.157449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a415d-0c92-42d4-80e9-201bd2736d16 None admin] RBAC: Authorizing `identity:get_user(user_id=572c94b6640b44cc8fb0508eda65e95d)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:20.159530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a415d-0c92-42d4-80e9-201bd2736d16 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:20.164538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 82/158] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:19 2022] GET /identity/v3/users/572c94b6640b44cc8fb0508eda65e95d => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:20.166298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3d38-0bcf-4e05-a3ad-4204909769da None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:20.166804 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3d38-0bcf-4e05-a3ad-4204909769da None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:20.167273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b3d38-0bcf-4e05-a3ad-4204909769da None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:20.168916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 83/159] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:20.170286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1457ac-5608-437c-9895-15889049d4b0 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:20.173784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1457ac-5608-437c-9895-15889049d4b0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:20.174380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1457ac-5608-437c-9895-15889049d4b0 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:20.175901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 84/160] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:20.180756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1457ac-5608-437c-9895-15889049d4b0 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:20.228759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b4388-ef5f-4536-9866-4e3815655ccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:20.230339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b4388-ef5f-4536-9866-4e3815655ccb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:20.258455 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:20.447344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1457ac-5608-437c-9895-15889049d4b0 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:20.462472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 77/161] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 23:35:20.464180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7093a2-b162-4984-95a1-28cb996866f4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:20.464797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7093a2-b162-4984-95a1-28cb996866f4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:20.465268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7093a2-b162-4984-95a1-28cb996866f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:20.501875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6e7093a2-b162-4984-95a1-28cb996866f4 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:20.503600 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6e7093a2-b162-4984-95a1-28cb996866f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:20.554254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e239a6-a90e-44a9-ad6c-a774f568a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:20.554927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e239a6-a90e-44a9-ad6c-a774f568a80e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:20.555215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e239a6-a90e-44a9-ad6c-a774f568a80e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:20.555500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e239a6-a90e-44a9-ad6c-a774f568a80e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:20.578724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:20.667700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 78/162] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:20.669673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213808-b385-4dc7-9c8a-0be5dbe580c2 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:20.670157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213808-b385-4dc7-9c8a-0be5dbe580c2 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:20.670621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213808-b385-4dc7-9c8a-0be5dbe580c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:20.712881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-81213808-b385-4dc7-9c8a-0be5dbe580c2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:20.714579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-81213808-b385-4dc7-9c8a-0be5dbe580c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:20.727904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e239a6-a90e-44a9-ad6c-a774f568a80e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:20.730172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e239a6-a90e-44a9-ad6c-a774f568a80e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:20.773268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:20.839428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 79/163] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:20.840930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213808-b385-4dc7-9c8a-0be5dbe580c2 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:20.966646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 85/164] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 790 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:20.967900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e239a6-a90e-44a9-ad6c-a774f568a80e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:21.016384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213808-b385-4dc7-9c8a-0be5dbe580c2 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:21.052624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:21.070935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556147b-91ac-44a2-a60c-50eaf1f59723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:21.072402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556147b-91ac-44a2-a60c-50eaf1f59723 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:21.073781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556147b-91ac-44a2-a60c-50eaf1f59723 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:21.076881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556147b-91ac-44a2-a60c-50eaf1f59723 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:21.079951 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556147b-91ac-44a2-a60c-50eaf1f59723 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:21.081824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556147b-91ac-44a2-a60c-50eaf1f59723 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:21.085395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 80/165] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:20 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:21.088201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556147b-91ac-44a2-a60c-50eaf1f59723 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:21.211485 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e239a6-a90e-44a9-ad6c-a774f568a80e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:21.214340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:21.274702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556147b-91ac-44a2-a60c-50eaf1f59723 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:21.278229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828884ce-0737-4141-964e-53308245d450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:21.279047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828884ce-0737-4141-964e-53308245d450 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:21.279331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828884ce-0737-4141-964e-53308245d450 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:21.279608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828884ce-0737-4141-964e-53308245d450 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:21.302048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828884ce-0737-4141-964e-53308245d450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-828884ce-0737-4141-964e-53308245d450 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:21.312350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:21.313257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:21.314530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:21.318016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 86/166] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:21.324036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828884ce-0737-4141-964e-53308245d450 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:21.326101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c784e4-d602-42c7-9bff-c61884496c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:21.327251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c784e4-d602-42c7-9bff-c61884496c8b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:21.327725 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c784e4-d602-42c7-9bff-c61884496c8b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:21.328187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c784e4-d602-42c7-9bff-c61884496c8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:21.339836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c784e4-d602-42c7-9bff-c61884496c8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-d6c784e4-d602-42c7-9bff-c61884496c8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:21.354569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:21.355451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:21.356201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:21.362431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 81/167] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:21.363522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c784e4-d602-42c7-9bff-c61884496c8b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:21.498731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:21.506975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828884ce-0737-4141-964e-53308245d450 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:21.550788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c784e4-d602-42c7-9bff-c61884496c8b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:21.563028 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:21.566288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595721-5061-42ee-a3ce-964e281f6653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:21.567298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595721-5061-42ee-a3ce-964e281f6653 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:21.567578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595721-5061-42ee-a3ce-964e281f6653 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:21.567860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595721-5061-42ee-a3ce-964e281f6653 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:21.569128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea595721-5061-42ee-a3ce-964e281f6653 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:21.570605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea595721-5061-42ee-a3ce-964e281f6653 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:21.572333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 87/168] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:21 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:21.573185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595721-5061-42ee-a3ce-964e281f6653 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:21.641302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b44e44-1694-4208-99f6-1e3bf9a839e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:21.642135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b44e44-1694-4208-99f6-1e3bf9a839e6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:21.642435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b44e44-1694-4208-99f6-1e3bf9a839e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:21.642897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b44e44-1694-4208-99f6-1e3bf9a839e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:21.643809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b44e44-1694-4208-99f6-1e3bf9a839e6 None admin] RBAC: Adding query filter params (role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8, user.id=572c94b6640b44cc8fb0508eda65e95d) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:21.644132 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b44e44-1694-4208-99f6-1e3bf9a839e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:21.645635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b44e44-1694-4208-99f6-1e3bf9a839e6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:21.662947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 82/169] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:21 2022] GET /identity/v3/role_assignments?user.id=572c94b6640b44cc8fb0508eda65e95d&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=86c49ec8587244e48fab81862ddb3641 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:21.663871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b44e44-1694-4208-99f6-1e3bf9a839e6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:21.823819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595721-5061-42ee-a3ce-964e281f6653 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:21.886012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b44e44-1694-4208-99f6-1e3bf9a839e6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:21.891118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144d3c6-dccc-479f-b800-5d045b4b5b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:21.892599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144d3c6-dccc-479f-b800-5d045b4b5b0b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:21.895333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144d3c6-dccc-479f-b800-5d045b4b5b0b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:21.896826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144d3c6-dccc-479f-b800-5d045b4b5b0b None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:21.898086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144d3c6-dccc-479f-b800-5d045b4b5b0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:21.902234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144d3c6-dccc-479f-b800-5d045b4b5b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:21.903914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144d3c6-dccc-479f-b800-5d045b4b5b0b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:21.937687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 88/170] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:21.939091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144d3c6-dccc-479f-b800-5d045b4b5b0b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:21.977277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d3236-6925-427c-8983-1d5e3c0392ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:21.979144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d3236-6925-427c-8983-1d5e3c0392ae None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:21.981350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d3236-6925-427c-8983-1d5e3c0392ae None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:21.982337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d3236-6925-427c-8983-1d5e3c0392ae None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:21.983292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d3236-6925-427c-8983-1d5e3c0392ae None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:21.984545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d3236-6925-427c-8983-1d5e3c0392ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:21.986877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d3236-6925-427c-8983-1d5e3c0392ae None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:22.004774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 83/171] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:22.007749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2613a7-5103-4631-ade2-953e7dc2d2ba None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.008964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2613a7-5103-4631-ade2-953e7dc2d2ba None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.010358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2613a7-5103-4631-ade2-953e7dc2d2ba None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.011927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:22.014124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 84/172] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:22.019552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2613a7-5103-4631-ade2-953e7dc2d2ba None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:22.085733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:22.168690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144d3c6-dccc-479f-b800-5d045b4b5b0b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:22.227050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292934-050a-4976-bd65-5aac8d90b64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:22.227835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292934-050a-4976-bd65-5aac8d90b64f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.228127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292934-050a-4976-bd65-5aac8d90b64f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.228411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292934-050a-4976-bd65-5aac8d90b64f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.229302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98292934-050a-4976-bd65-5aac8d90b64f None admin] RBAC: Adding query filter params (role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8, user.id=572c94b6640b44cc8fb0508eda65e95d) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:22.229623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98292934-050a-4976-bd65-5aac8d90b64f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:22.229891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2613a7-5103-4631-ade2-953e7dc2d2ba None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:22.231076 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98292934-050a-4976-bd65-5aac8d90b64f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:22.237917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 89/173] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:21 2022] GET /identity/v3/role_assignments?user.id=572c94b6640b44cc8fb0508eda65e95d&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:22.238736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292934-050a-4976-bd65-5aac8d90b64f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:22.269615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb4b54-aa42-4cfd-82fe-83899ddaa5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:22.270899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb4b54-aa42-4cfd-82fe-83899ddaa5a0 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.271404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb4b54-aa42-4cfd-82fe-83899ddaa5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.271970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb4b54-aa42-4cfd-82fe-83899ddaa5a0 None admin] PATH_INFO: `/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.273921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb4b54-aa42-4cfd-82fe-83899ddaa5a0 None admin] RBAC: Authorizing `identity:get_user(user_id=19e6b8d9d8f94b0c8f316b8b2060e76d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:22.275748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb4b54-aa42-4cfd-82fe-83899ddaa5a0 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:22.277903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 85/174] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:22 2022] GET /identity/v3/users/19e6b8d9d8f94b0c8f316b8b2060e76d => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:22.279535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ab891-cd5b-4570-ba87-0699f0ee8ae2 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.280064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ab891-cd5b-4570-ba87-0699f0ee8ae2 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.280654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ab891-cd5b-4570-ba87-0699f0ee8ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.300256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-834ab891-cd5b-4570-ba87-0699f0ee8ae2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:22.301972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-834ab891-cd5b-4570-ba87-0699f0ee8ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:22.350836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:22.358141 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292934-050a-4976-bd65-5aac8d90b64f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:22.381383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 86/175] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:22.390397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447617d7-12db-4e71-b14d-e0109a75b22f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.390927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447617d7-12db-4e71-b14d-e0109a75b22f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.391418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447617d7-12db-4e71-b14d-e0109a75b22f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.393242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 87/176] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:22.394774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77fb93-a4c2-48d3-bf9d-5c97715540dc None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.395323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77fb93-a4c2-48d3-bf9d-5c97715540dc None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.395793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77fb93-a4c2-48d3-bf9d-5c97715540dc None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.397569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 88/177] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:22.400192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77fb93-a4c2-48d3-bf9d-5c97715540dc None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:22.402138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f4132-36ba-4e29-8f61-5a6a2f155553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:22.402793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f4132-36ba-4e29-8f61-5a6a2f155553 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.403175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f4132-36ba-4e29-8f61-5a6a2f155553 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.403458 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f4132-36ba-4e29-8f61-5a6a2f155553 None admin] PATH_INFO: `/v3/users/572c94b6640b44cc8fb0508eda65e95d` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.404790 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f4132-36ba-4e29-8f61-5a6a2f155553 None admin] RBAC: Authorizing `identity:get_user(user_id=572c94b6640b44cc8fb0508eda65e95d)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:22.407345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f4132-36ba-4e29-8f61-5a6a2f155553 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:22.408518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 90/178] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:22 2022] GET /identity/v3/users/572c94b6640b44cc8fb0508eda65e95d => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:22.412872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ce5d2-c3fd-4e64-82d5-cb2e2cb3e476 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.412872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ce5d2-c3fd-4e64-82d5-cb2e2cb3e476 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.412872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98ce5d2-c3fd-4e64-82d5-cb2e2cb3e476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.430383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d98ce5d2-c3fd-4e64-82d5-cb2e2cb3e476 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:22.431586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d98ce5d2-c3fd-4e64-82d5-cb2e2cb3e476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:22.444193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:22.515491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 91/179] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:22.516659 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b7009-18f2-4c56-b393-a31c99723b95 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.517504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b7009-18f2-4c56-b393-a31c99723b95 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.517770 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b7009-18f2-4c56-b393-a31c99723b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.535083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-449b7009-18f2-4c56-b393-a31c99723b95 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:22.536273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-449b7009-18f2-4c56-b393-a31c99723b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:22.538806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77fb93-a4c2-48d3-bf9d-5c97715540dc None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:22.572376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c06e6-f11d-4e69-bc8d-5ec6c4419b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:22.573064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c06e6-f11d-4e69-bc8d-5ec6c4419b01 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.573345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c06e6-f11d-4e69-bc8d-5ec6c4419b01 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.573624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c06e6-f11d-4e69-bc8d-5ec6c4419b01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.604106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 92/180] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:22.604940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b7009-18f2-4c56-b393-a31c99723b95 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:22.628362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:22.647105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c06e6-f11d-4e69-bc8d-5ec6c4419b01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:22.648869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c06e6-f11d-4e69-bc8d-5ec6c4419b01 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:22.675284 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:22.702368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b7009-18f2-4c56-b393-a31c99723b95 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:22.739142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25e32d-0ef9-4dd9-b4a0-69e6fc4ce933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:22.740182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25e32d-0ef9-4dd9-b4a0-69e6fc4ce933 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.740677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25e32d-0ef9-4dd9-b4a0-69e6fc4ce933 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.741157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25e32d-0ef9-4dd9-b4a0-69e6fc4ce933 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.747779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 89/181] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:22.748645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c06e6-f11d-4e69-bc8d-5ec6c4419b01 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:22.811478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25e32d-0ef9-4dd9-b4a0-69e6fc4ce933 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:22.812994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25e32d-0ef9-4dd9-b4a0-69e6fc4ce933 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:22.835055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:22.844494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c06e6-f11d-4e69-bc8d-5ec6c4419b01 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:22.871006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:22.880138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b115ff6-8070-4f74-9041-116cbf020fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:22.880969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b115ff6-8070-4f74-9041-116cbf020fc3 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.881253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b115ff6-8070-4f74-9041-116cbf020fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.881528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b115ff6-8070-4f74-9041-116cbf020fc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.891544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b115ff6-8070-4f74-9041-116cbf020fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-8b115ff6-8070-4f74-9041-116cbf020fc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:22.899441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:22.900299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:22.901396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 90/182] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:22.906634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16abe0-3f78-4082-bea7-d1b53764689d None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:22.906918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16abe0-3f78-4082-bea7-d1b53764689d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:22.907196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16abe0-3f78-4082-bea7-d1b53764689d None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:22.909414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 91/183] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:22.910054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16abe0-3f78-4082-bea7-d1b53764689d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:22.910514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 93/184] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:22.911354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25e32d-0ef9-4dd9-b4a0-69e6fc4ce933 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.012384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25e32d-0ef9-4dd9-b4a0-69e6fc4ce933 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.020965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16abe0-3f78-4082-bea7-d1b53764689d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.034958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:23.047004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f31b1-0887-4632-a8c6-cd6fac9f2c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.047799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f31b1-0887-4632-a8c6-cd6fac9f2c60 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.048078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f31b1-0887-4632-a8c6-cd6fac9f2c60 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.048458 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f31b1-0887-4632-a8c6-cd6fac9f2c60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.049665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f31b1-0887-4632-a8c6-cd6fac9f2c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:23.051106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f31b1-0887-4632-a8c6-cd6fac9f2c60 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:23.052735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 94/185] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:23.053494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f31b1-0887-4632-a8c6-cd6fac9f2c60 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.054820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bb111-28e9-4872-bb4d-1fed1dbb0323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.055615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bb111-28e9-4872-bb4d-1fed1dbb0323 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.055901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bb111-28e9-4872-bb4d-1fed1dbb0323 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.056172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bb111-28e9-4872-bb4d-1fed1dbb0323 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.066932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bb111-28e9-4872-bb4d-1fed1dbb0323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-4f2bb111-28e9-4872-bb4d-1fed1dbb0323 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:23.074819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:23.075698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:23.076448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 92/186] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:23.077113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bb111-28e9-4872-bb4d-1fed1dbb0323 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.128388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:23.152106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f31b1-0887-4632-a8c6-cd6fac9f2c60 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.178952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bb111-28e9-4872-bb4d-1fed1dbb0323 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.185121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a830fa9-07ba-4729-8fa4-5039e3c7c920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.185988 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a830fa9-07ba-4729-8fa4-5039e3c7c920 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.186271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a830fa9-07ba-4729-8fa4-5039e3c7c920 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.186556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a830fa9-07ba-4729-8fa4-5039e3c7c920 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.187754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a830fa9-07ba-4729-8fa4-5039e3c7c920 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:23.189999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a830fa9-07ba-4729-8fa4-5039e3c7c920 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:23.192239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 95/187] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:23.195065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b631fb-1109-4754-9eda-dd86a382ea14 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.195356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b631fb-1109-4754-9eda-dd86a382ea14 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.196940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b631fb-1109-4754-9eda-dd86a382ea14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.212741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed26f2-0456-42cf-a0e2-232247f4d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.213423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed26f2-0456-42cf-a0e2-232247f4d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.213709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed26f2-0456-42cf-a0e2-232247f4d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.213986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed26f2-0456-42cf-a0e2-232247f4d1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.214745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed26f2-0456-42cf-a0e2-232247f4d1d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:23.214874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a4b631fb-1109-4754-9eda-dd86a382ea14 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:23.215055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed26f2-0456-42cf-a0e2-232247f4d1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:23.216039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a4b631fb-1109-4754-9eda-dd86a382ea14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:23.216452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed26f2-0456-42cf-a0e2-232247f4d1d8 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:23.224739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 93/188] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:23.224739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed26f2-0456-42cf-a0e2-232247f4d1d8 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.274477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:23.288022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 96/189] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:23.288844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b631fb-1109-4754-9eda-dd86a382ea14 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.324911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed26f2-0456-42cf-a0e2-232247f4d1d8 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.336863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:23.360493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c7b02-97a2-4224-90fd-c91c540db52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.362357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c7b02-97a2-4224-90fd-c91c540db52c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.362645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c7b02-97a2-4224-90fd-c91c540db52c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.362925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c7b02-97a2-4224-90fd-c91c540db52c None admin] PATH_INFO: `/v3/projects/1b89af06fd92444589c10d61e5cbc1aa` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.364108 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c7b02-97a2-4224-90fd-c91c540db52c None admin] RBAC: Authorizing `identity:get_project(project_id=1b89af06fd92444589c10d61e5cbc1aa)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:23.365616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c7b02-97a2-4224-90fd-c91c540db52c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:23.367526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 94/190] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/projects/1b89af06fd92444589c10d61e5cbc1aa => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:23.368195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c7b02-97a2-4224-90fd-c91c540db52c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.395357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b631fb-1109-4754-9eda-dd86a382ea14 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.430493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fb5a-7b51-479f-9a43-8fb7f6104fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.431162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fb5a-7b51-479f-9a43-8fb7f6104fe4 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.431463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fb5a-7b51-479f-9a43-8fb7f6104fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.431785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fb5a-7b51-479f-9a43-8fb7f6104fe4 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.432503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12fb5a-7b51-479f-9a43-8fb7f6104fe4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:23.433967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12fb5a-7b51-479f-9a43-8fb7f6104fe4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:23.453991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 97/191] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:35:23 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:35:23.455509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fb5a-7b51-479f-9a43-8fb7f6104fe4 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.467925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c7b02-97a2-4224-90fd-c91c540db52c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.505270 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:23.511295 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3df17a-3fab-491e-add1-2b2be6514471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.512090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3df17a-3fab-491e-add1-2b2be6514471 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.512452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3df17a-3fab-491e-add1-2b2be6514471 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.512773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3df17a-3fab-491e-add1-2b2be6514471 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.513554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3df17a-3fab-491e-add1-2b2be6514471 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:23.513868 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3df17a-3fab-491e-add1-2b2be6514471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:23.515292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3df17a-3fab-491e-add1-2b2be6514471 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:23.528705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 95/192] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:23.528705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3df17a-3fab-491e-add1-2b2be6514471 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.576313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:23.586398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fb5a-7b51-479f-9a43-8fb7f6104fe4 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.630693 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e216dc3-dd86-4f2b-b531-2dc34b4597cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.631372 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e216dc3-dd86-4f2b-b531-2dc34b4597cd None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.631660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e216dc3-dd86-4f2b-b531-2dc34b4597cd None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.631938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e216dc3-dd86-4f2b-b531-2dc34b4597cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.632825 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e216dc3-dd86-4f2b-b531-2dc34b4597cd None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8, user.id=19e6b8d9d8f94b0c8f316b8b2060e76d) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:23.633207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e216dc3-dd86-4f2b-b531-2dc34b4597cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:23.634657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e216dc3-dd86-4f2b-b531-2dc34b4597cd None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:23.640825 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 98/193] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/role_assignments?user.id=19e6b8d9d8f94b0c8f316b8b2060e76d&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:23.641638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e216dc3-dd86-4f2b-b531-2dc34b4597cd None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.651664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3df17a-3fab-491e-add1-2b2be6514471 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.696938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eac13-3711-4dfa-856e-296b50f8d75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.697850 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eac13-3711-4dfa-856e-296b50f8d75b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.698139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eac13-3711-4dfa-856e-296b50f8d75b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.698421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eac13-3711-4dfa-856e-296b50f8d75b None admin] PATH_INFO: `/v3/users/159fd9f414d1444ca9249426decfb3b1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.715730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eac13-3711-4dfa-856e-296b50f8d75b None admin] RBAC: Authorizing `identity:get_user(user_id=159fd9f414d1444ca9249426decfb3b1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:23.717360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eac13-3711-4dfa-856e-296b50f8d75b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:23.720309 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 96/194] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/users/159fd9f414d1444ca9249426decfb3b1 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:23.721186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eac13-3711-4dfa-856e-296b50f8d75b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.793050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e216dc3-dd86-4f2b-b531-2dc34b4597cd None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.806344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:23.842954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3926e9-c3dc-4b08-a822-75e2fa44b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.843801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3926e9-c3dc-4b08-a822-75e2fa44b558 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.844086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3926e9-c3dc-4b08-a822-75e2fa44b558 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.844361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3926e9-c3dc-4b08-a822-75e2fa44b558 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.858114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3926e9-c3dc-4b08-a822-75e2fa44b558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:23.860633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:23.866009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eac13-3711-4dfa-856e-296b50f8d75b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-0c3926e9-c3dc-4b08-a822-75e2fa44b558 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:23.867394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:23.868411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:23.869304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 99/195] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:23.870078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3926e9-c3dc-4b08-a822-75e2fa44b558 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:23.919363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76084b9-67bf-4958-b4d4-12e4aa5eabf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:23.919968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76084b9-67bf-4958-b4d4-12e4aa5eabf2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:23.920308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76084b9-67bf-4958-b4d4-12e4aa5eabf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:23.920620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76084b9-67bf-4958-b4d4-12e4aa5eabf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:23.921493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76084b9-67bf-4958-b4d4-12e4aa5eabf2 None admin] RBAC: Adding query filter params (role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=1b89af06fd92444589c10d61e5cbc1aa, user.id=572c94b6640b44cc8fb0508eda65e95d) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:23.921809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76084b9-67bf-4958-b4d4-12e4aa5eabf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:23.925674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76084b9-67bf-4958-b4d4-12e4aa5eabf2 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:23.936661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 97/196] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/role_assignments?user.id=572c94b6640b44cc8fb0508eda65e95d&scope.project.id=1b89af06fd92444589c10d61e5cbc1aa&role.id=86c49ec8587244e48fab81862ddb3641 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:23.937861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76084b9-67bf-4958-b4d4-12e4aa5eabf2 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:24.028509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3926e9-c3dc-4b08-a822-75e2fa44b558 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:24.059461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76084b9-67bf-4958-b4d4-12e4aa5eabf2 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:24.075536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd36df3-283e-488b-91c2-6d394516f5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:24.076671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd36df3-283e-488b-91c2-6d394516f5c2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:24.077031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd36df3-283e-488b-91c2-6d394516f5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:24.077407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd36df3-283e-488b-91c2-6d394516f5c2 None admin] PATH_INFO: `/v3/users/b598f40388594a62814f9f05eba8aad3` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:24.081341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd36df3-283e-488b-91c2-6d394516f5c2 None admin] RBAC: Authorizing `identity:get_user(user_id=b598f40388594a62814f9f05eba8aad3)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:24.083062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd36df3-283e-488b-91c2-6d394516f5c2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:24.085719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 100/197] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/users/b598f40388594a62814f9f05eba8aad3 => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:24.086257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd36df3-283e-488b-91c2-6d394516f5c2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:24.101622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b73b0-e5ae-44d4-9ff4-ef225c4a015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:24.102835 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b73b0-e5ae-44d4-9ff4-ef225c4a015a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:24.103387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b73b0-e5ae-44d4-9ff4-ef225c4a015a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:24.104154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b73b0-e5ae-44d4-9ff4-ef225c4a015a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:24.134692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:24.141503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:24.187637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b73b0-e5ae-44d4-9ff4-ef225c4a015a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:24.189659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b73b0-e5ae-44d4-9ff4-ef225c4a015a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:24.213526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd36df3-283e-488b-91c2-6d394516f5c2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:24.253248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232ef67-24ff-48fb-bc52-fab3315affd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:24.254093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232ef67-24ff-48fb-bc52-fab3315affd3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:24.254875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232ef67-24ff-48fb-bc52-fab3315affd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:24.254875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232ef67-24ff-48fb-bc52-fab3315affd3 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:24.255749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232ef67-24ff-48fb-bc52-fab3315affd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:24.256132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232ef67-24ff-48fb-bc52-fab3315affd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:24.257778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232ef67-24ff-48fb-bc52-fab3315affd3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:24.268699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 101/198] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:24.273910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232ef67-24ff-48fb-bc52-fab3315affd3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:24.309470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 98/199] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:24.310659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b73b0-e5ae-44d4-9ff4-ef225c4a015a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:24.400961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:24.401388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:24.404440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232ef67-24ff-48fb-bc52-fab3315affd3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:24.425401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b73b0-e5ae-44d4-9ff4-ef225c4a015a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:24.444867 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4469d08-f7a2-4a03-bbb0-70b1baf59305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:24.445667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4469d08-f7a2-4a03-bbb0-70b1baf59305 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:24.446023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4469d08-f7a2-4a03-bbb0-70b1baf59305 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:24.446368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4469d08-f7a2-4a03-bbb0-70b1baf59305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:24.463875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79190ece-c696-45f6-83c9-4d9fecf9cbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:24.464933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79190ece-c696-45f6-83c9-4d9fecf9cbe0 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:24.465328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79190ece-c696-45f6-83c9-4d9fecf9cbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:24.465707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79190ece-c696-45f6-83c9-4d9fecf9cbe0 None admin] PATH_INFO: `/v3/users/159fd9f414d1444ca9249426decfb3b1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:24.467045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79190ece-c696-45f6-83c9-4d9fecf9cbe0 None admin] RBAC: Authorizing `identity:get_user(user_id=159fd9f414d1444ca9249426decfb3b1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:24.468658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79190ece-c696-45f6-83c9-4d9fecf9cbe0 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:24.471463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 99/200] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:24 2022] GET /identity/v3/users/159fd9f414d1444ca9249426decfb3b1 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:24.472424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79190ece-c696-45f6-83c9-4d9fecf9cbe0 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:24.525202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4469d08-f7a2-4a03-bbb0-70b1baf59305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:24.526890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4469d08-f7a2-4a03-bbb0-70b1baf59305 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:24.584016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79190ece-c696-45f6-83c9-4d9fecf9cbe0 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:24.617570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:24.623880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e83ac8-b141-410b-9e6e-6eb55e9b817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:24.624943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e83ac8-b141-410b-9e6e-6eb55e9b817c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:24.625205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e83ac8-b141-410b-9e6e-6eb55e9b817c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:24.625490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e83ac8-b141-410b-9e6e-6eb55e9b817c None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:24.626918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e83ac8-b141-410b-9e6e-6eb55e9b817c None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:24.628692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e83ac8-b141-410b-9e6e-6eb55e9b817c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:24.635849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 100/201] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:24 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:24.636554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e83ac8-b141-410b-9e6e-6eb55e9b817c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:24.650014 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 102/202] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:24.650534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:24.652036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4469d08-f7a2-4a03-bbb0-70b1baf59305 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:24.749948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e83ac8-b141-410b-9e6e-6eb55e9b817c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:24.777043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4469d08-f7a2-4a03-bbb0-70b1baf59305 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:24.792946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17d2e5-4a0b-4d0c-9882-ce50385ba454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:24.793803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17d2e5-4a0b-4d0c-9882-ce50385ba454 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:24.794221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17d2e5-4a0b-4d0c-9882-ce50385ba454 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:24.794631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17d2e5-4a0b-4d0c-9882-ce50385ba454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:24.816685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd2ebe-1317-42d0-8b35-3eeb6f52376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:24.817421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd2ebe-1317-42d0-8b35-3eeb6f52376a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:24.817712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd2ebe-1317-42d0-8b35-3eeb6f52376a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:24.817993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd2ebe-1317-42d0-8b35-3eeb6f52376a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:24.819239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd2ebe-1317-42d0-8b35-3eeb6f52376a None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=9b25ac538fb448279a63db11f84f70de, user.id=159fd9f414d1444ca9249426decfb3b1) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:24.819366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd2ebe-1317-42d0-8b35-3eeb6f52376a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:24.820872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd2ebe-1317-42d0-8b35-3eeb6f52376a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:24.826475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 103/203] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:24 2022] GET /identity/v3/role_assignments?user.id=159fd9f414d1444ca9249426decfb3b1&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:24.827330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd2ebe-1317-42d0-8b35-3eeb6f52376a None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:24.868845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:24.873601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17d2e5-4a0b-4d0c-9882-ce50385ba454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:24.875348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17d2e5-4a0b-4d0c-9882-ce50385ba454 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:24.971890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:24.997085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 101/204] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:25.005767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17d2e5-4a0b-4d0c-9882-ce50385ba454 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:25.069806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd2ebe-1317-42d0-8b35-3eeb6f52376a None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:25.112195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b472496-b204-4805-9107-6ef04d1dce36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:25.113130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b472496-b204-4805-9107-6ef04d1dce36 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:25.113419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b472496-b204-4805-9107-6ef04d1dce36 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:25.113700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b472496-b204-4805-9107-6ef04d1dce36 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:25.114976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b472496-b204-4805-9107-6ef04d1dce36 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:25.116458 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b472496-b204-4805-9107-6ef04d1dce36 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:25.118212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 104/205] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:24 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:25.125523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b472496-b204-4805-9107-6ef04d1dce36 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:25.166154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:25.178524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17d2e5-4a0b-4d0c-9882-ce50385ba454 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:25.246253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fd2d3-05f2-4b40-9bd0-0b689ab0ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:25.247129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fd2d3-05f2-4b40-9bd0-0b689ab0ecc6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:25.247411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fd2d3-05f2-4b40-9bd0-0b689ab0ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:25.247690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fd2d3-05f2-4b40-9bd0-0b689ab0ecc6 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:25.249977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fd2d3-05f2-4b40-9bd0-0b689ab0ecc6 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:25.251973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fd2d3-05f2-4b40-9bd0-0b689ab0ecc6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:25.254844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 102/206] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:25 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:25.261226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fd2d3-05f2-4b40-9bd0-0b689ab0ecc6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:25.263935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b472496-b204-4805-9107-6ef04d1dce36 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:25.327949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:25.349830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaedafc-2fec-444a-861b-ead7f191887d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:25.350494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaedafc-2fec-444a-861b-ead7f191887d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:25.350772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaedafc-2fec-444a-861b-ead7f191887d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:25.351053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaedafc-2fec-444a-861b-ead7f191887d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:25.351895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaedafc-2fec-444a-861b-ead7f191887d None admin] RBAC: Adding query filter params (role.id=3f15101d95de479cacc1f6d91a2ff504, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8, user.id=b598f40388594a62814f9f05eba8aad3) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:25.352206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaedafc-2fec-444a-861b-ead7f191887d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:25.353655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaedafc-2fec-444a-861b-ead7f191887d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:25.367942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 105/207] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:25 2022] GET /identity/v3/role_assignments?user.id=b598f40388594a62814f9f05eba8aad3&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=3f15101d95de479cacc1f6d91a2ff504 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:25.379176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fd2d3-05f2-4b40-9bd0-0b689ab0ecc6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:25.418528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712a5c3-0843-40f4-96c7-305530889300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:25.419569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712a5c3-0843-40f4-96c7-305530889300 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:25.420054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712a5c3-0843-40f4-96c7-305530889300 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:25.420538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712a5c3-0843-40f4-96c7-305530889300 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:25.421666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712a5c3-0843-40f4-96c7-305530889300 None admin] RBAC: Adding query filter params (role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=9b25ac538fb448279a63db11f84f70de, user.id=159fd9f414d1444ca9249426decfb3b1) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:25.422235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712a5c3-0843-40f4-96c7-305530889300 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:25.424408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712a5c3-0843-40f4-96c7-305530889300 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:25.431322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 103/208] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:25 2022] GET /identity/v3/role_assignments?user.id=159fd9f414d1444ca9249426decfb3b1&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:26.062113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d233e9-a145-4222-a73e-965795617ec5 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.062477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d233e9-a145-4222-a73e-965795617ec5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.062668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d233e9-a145-4222-a73e-965795617ec5 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.064371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 106/209] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:26.069666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae19a4-ef99-47cb-bc78-0070dad8fd11 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.070183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae19a4-ef99-47cb-bc78-0070dad8fd11 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.070676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae19a4-ef99-47cb-bc78-0070dad8fd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.087972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d9ae19a4-ef99-47cb-bc78-0070dad8fd11 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:26.089495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d9ae19a4-ef99-47cb-bc78-0070dad8fd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:26.115623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:26.158866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda0e1-a22a-4a72-b170-180a7e082d17 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.159363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda0e1-a22a-4a72-b170-180a7e082d17 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.159470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda0e1-a22a-4a72-b170-180a7e082d17 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.160808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 107/210] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:26.166101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd8139-7f26-49a4-82a7-8c383a80bc2c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.166509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd8139-7f26-49a4-82a7-8c383a80bc2c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.166886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd8139-7f26-49a4-82a7-8c383a80bc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.168750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 104/211] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:26.184664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-91fd8139-7f26-49a4-82a7-8c383a80bc2c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:26.185842 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-91fd8139-7f26-49a4-82a7-8c383a80bc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:26.193306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae19a4-ef99-47cb-bc78-0070dad8fd11 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:26.264479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 108/212] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:26.290399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd8139-7f26-49a4-82a7-8c383a80bc2c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:26.308951 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae19a4-ef99-47cb-bc78-0070dad8fd11 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:26.358142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc82958-f4b9-492f-bc58-704c6ec33cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:26.359211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc82958-f4b9-492f-bc58-704c6ec33cec None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.359927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc82958-f4b9-492f-bc58-704c6ec33cec None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.360409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc82958-f4b9-492f-bc58-704c6ec33cec None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.375553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc82958-f4b9-492f-bc58-704c6ec33cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-fcc82958-f4b9-492f-bc58-704c6ec33cec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:26.386272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:26.387129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:26.387876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:26.395771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 105/213] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:26.396907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc82958-f4b9-492f-bc58-704c6ec33cec None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:26.406805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd8139-7f26-49a4-82a7-8c383a80bc2c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:26.426467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:26.443704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:26.459472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbe847-59f3-4aac-9bac-121fddf2afbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:26.460554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbe847-59f3-4aac-9bac-121fddf2afbe None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.461593 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbe847-59f3-4aac-9bac-121fddf2afbe None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.462073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbe847-59f3-4aac-9bac-121fddf2afbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.463716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbe847-59f3-4aac-9bac-121fddf2afbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:26.466315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbe847-59f3-4aac-9bac-121fddf2afbe None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:26.468984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 109/214] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:26.474831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbe847-59f3-4aac-9bac-121fddf2afbe None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:26.517945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc82958-f4b9-492f-bc58-704c6ec33cec None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:26.554882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26beb36d-f0ec-4314-9663-30a460a71b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:26.555559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beb36d-f0ec-4314-9663-30a460a71b55 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.555899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beb36d-f0ec-4314-9663-30a460a71b55 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.556182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26beb36d-f0ec-4314-9663-30a460a71b55 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.557033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beb36d-f0ec-4314-9663-30a460a71b55 None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:26.557348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beb36d-f0ec-4314-9663-30a460a71b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:26.558866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26beb36d-f0ec-4314-9663-30a460a71b55 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:26.568671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 106/215] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:26.574171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26beb36d-f0ec-4314-9663-30a460a71b55 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:26.587261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbe847-59f3-4aac-9bac-121fddf2afbe None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:26.628260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796b4d7-a3b9-4e32-8c89-65d281f965d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:26.629409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796b4d7-a3b9-4e32-8c89-65d281f965d4 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.629898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796b4d7-a3b9-4e32-8c89-65d281f965d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.630401 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796b4d7-a3b9-4e32-8c89-65d281f965d4 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.631434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796b4d7-a3b9-4e32-8c89-65d281f965d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:26.633311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7796b4d7-a3b9-4e32-8c89-65d281f965d4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:26.669047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:26.670126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 110/216] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:35:26 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:35:26.692256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26beb36d-f0ec-4314-9663-30a460a71b55 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:26.731941 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f985e-eb1d-405f-b32b-f5596ec98293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:26.732871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f985e-eb1d-405f-b32b-f5596ec98293 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.733204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f985e-eb1d-405f-b32b-f5596ec98293 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.733486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f985e-eb1d-405f-b32b-f5596ec98293 None admin] PATH_INFO: `/v3/users/e45c1ccf481845c7a84a4d2f4113a84c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.749291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f985e-eb1d-405f-b32b-f5596ec98293 None admin] RBAC: Authorizing `identity:get_user(user_id=e45c1ccf481845c7a84a4d2f4113a84c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:26.750886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f985e-eb1d-405f-b32b-f5596ec98293 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:26.753618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 107/217] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:26 2022] GET /identity/v3/users/e45c1ccf481845c7a84a4d2f4113a84c => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:26.759544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796b4d7-a3b9-4e32-8c89-65d281f965d4 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:26.815592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:26.874586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796b4d7-a3b9-4e32-8c89-65d281f965d4 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:26.913390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17abc26-6d0b-4475-8363-ede302b47212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:26.914515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17abc26-6d0b-4475-8363-ede302b47212 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:26.915044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17abc26-6d0b-4475-8363-ede302b47212 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:26.915618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17abc26-6d0b-4475-8363-ede302b47212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:26.998685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17abc26-6d0b-4475-8363-ede302b47212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:27.000180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17abc26-6d0b-4475-8363-ede302b47212 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:27.059160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:27.109040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 111/218] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:27.115202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f985e-eb1d-405f-b32b-f5596ec98293 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:27.216895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fd9e7-c63f-4121-99db-d0b8a7f28353 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:27.217430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fd9e7-c63f-4121-99db-d0b8a7f28353 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:27.217430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fd9e7-c63f-4121-99db-d0b8a7f28353 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:27.218949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 112/219] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:35:27.224486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44412e53-76ff-4870-b965-5f69125191f7 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:27.224803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44412e53-76ff-4870-b965-5f69125191f7 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:27.225083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44412e53-76ff-4870-b965-5f69125191f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:27.227268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f985e-eb1d-405f-b32b-f5596ec98293 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:27.254013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-44412e53-76ff-4870-b965-5f69125191f7 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:27.255222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-44412e53-76ff-4870-b965-5f69125191f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:27.317107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46201ed2-c9d3-4cac-9f36-f8941fd41e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:27.317985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46201ed2-c9d3-4cac-9f36-f8941fd41e6a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:27.318263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46201ed2-c9d3-4cac-9f36-f8941fd41e6a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:27.318534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46201ed2-c9d3-4cac-9f36-f8941fd41e6a None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:27.319964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46201ed2-c9d3-4cac-9f36-f8941fd41e6a None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:27.322560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46201ed2-c9d3-4cac-9f36-f8941fd41e6a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:27.324445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 108/220] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:27 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:27.331543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46201ed2-c9d3-4cac-9f36-f8941fd41e6a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:27.346584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:27.412441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 113/221] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 24 23:35:27.469438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44412e53-76ff-4870-b965-5f69125191f7 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:27.514562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46201ed2-c9d3-4cac-9f36-f8941fd41e6a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:27.561257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:27.581417 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44412e53-76ff-4870-b965-5f69125191f7 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:27.589500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809cf21-ff05-45f5-b349-4ffa532ad954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:27.590226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809cf21-ff05-45f5-b349-4ffa532ad954 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:27.590509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809cf21-ff05-45f5-b349-4ffa532ad954 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:27.590784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809cf21-ff05-45f5-b349-4ffa532ad954 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:27.591657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809cf21-ff05-45f5-b349-4ffa532ad954 None admin] RBAC: Adding query filter params (role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=9b25ac538fb448279a63db11f84f70de, user.id=e45c1ccf481845c7a84a4d2f4113a84c) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:27.591968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809cf21-ff05-45f5-b349-4ffa532ad954 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:27.593438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809cf21-ff05-45f5-b349-4ffa532ad954 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:27.614178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 109/222] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:27 2022] GET /identity/v3/role_assignments?user.id=e45c1ccf481845c7a84a4d2f4113a84c&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=86c49ec8587244e48fab81862ddb3641 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:27.640683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25169b56-41f3-4ee9-8fca-35e3401d0821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:27.641507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25169b56-41f3-4ee9-8fca-35e3401d0821 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:27.642661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25169b56-41f3-4ee9-8fca-35e3401d0821 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:27.642955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25169b56-41f3-4ee9-8fca-35e3401d0821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:27.662949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25169b56-41f3-4ee9-8fca-35e3401d0821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-25169b56-41f3-4ee9-8fca-35e3401d0821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:27.684975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:27.685883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:27.686653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 114/223] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:27.693490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809cf21-ff05-45f5-b349-4ffa532ad954 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:27.763835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b22dc2-c706-4ecd-a840-4d3df46020fe None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:27.764163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b22dc2-c706-4ecd-a840-4d3df46020fe None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:27.764385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b22dc2-c706-4ecd-a840-4d3df46020fe None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:27.765870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 115/224] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:35:27.778822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffb9ae-271b-49e9-87be-100967c94df9 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:27.779147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffb9ae-271b-49e9-87be-100967c94df9 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:27.779384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffb9ae-271b-49e9-87be-100967c94df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:27.794668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:27.817894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-24ffb9ae-271b-49e9-87be-100967c94df9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:27.819250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-24ffb9ae-271b-49e9-87be-100967c94df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:27.841771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809cf21-ff05-45f5-b349-4ffa532ad954 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:27.916508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde9425-61ea-41cc-be08-60f4f7ed2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:27.917214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde9425-61ea-41cc-be08-60f4f7ed2f04 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:27.917497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde9425-61ea-41cc-be08-60f4f7ed2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:27.917776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde9425-61ea-41cc-be08-60f4f7ed2f04 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:27.918584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde9425-61ea-41cc-be08-60f4f7ed2f04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:27.918840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde9425-61ea-41cc-be08-60f4f7ed2f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:27.920227 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde9425-61ea-41cc-be08-60f4f7ed2f04 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:27.932964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 110/225] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:27.937089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 116/226] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:27.945642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde9425-61ea-41cc-be08-60f4f7ed2f04 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:27.975922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffb9ae-271b-49e9-87be-100967c94df9 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:28.070871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde9425-61ea-41cc-be08-60f4f7ed2f04 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:28.127138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe78be-5cf4-4ecd-a514-beee8521b616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:28.128044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe78be-5cf4-4ecd-a514-beee8521b616 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:28.128326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe78be-5cf4-4ecd-a514-beee8521b616 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:28.128618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe78be-5cf4-4ecd-a514-beee8521b616 None admin] PATH_INFO: `/v3/users/159fd9f414d1444ca9249426decfb3b1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:28.129984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe78be-5cf4-4ecd-a514-beee8521b616 None admin] RBAC: Authorizing `identity:get_user(user_id=159fd9f414d1444ca9249426decfb3b1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:28.131438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe78be-5cf4-4ecd-a514-beee8521b616 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:28.133287 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 111/227] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:27 2022] GET /identity/v3/users/159fd9f414d1444ca9249426decfb3b1 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:28.140120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe78be-5cf4-4ecd-a514-beee8521b616 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:28.141557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:28.160949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:28.201036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffb9ae-271b-49e9-87be-100967c94df9 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:28.262085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a56ff-9734-4541-9734-66f647ded694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:28.262914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a56ff-9734-4541-9734-66f647ded694 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:28.263196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a56ff-9734-4541-9734-66f647ded694 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:28.263471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a56ff-9734-4541-9734-66f647ded694 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:28.286140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a56ff-9734-4541-9734-66f647ded694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-8b3a56ff-9734-4541-9734-66f647ded694 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:28.294978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:28.295880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:28.297085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 117/228] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:28.303642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe78be-5cf4-4ecd-a514-beee8521b616 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:28.307069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a56ff-9734-4541-9734-66f647ded694 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:28.378360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae91477-9137-4459-8310-86c3f590c7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:28.379233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae91477-9137-4459-8310-86c3f590c7cc None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:28.379517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae91477-9137-4459-8310-86c3f590c7cc None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:28.379791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae91477-9137-4459-8310-86c3f590c7cc None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:28.381309 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae91477-9137-4459-8310-86c3f590c7cc None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:28.382776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae91477-9137-4459-8310-86c3f590c7cc None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:28.384483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 112/229] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:28 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:28.392079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae91477-9137-4459-8310-86c3f590c7cc None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:28.484793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a56ff-9734-4541-9734-66f647ded694 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:28.513389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:28.530632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:28.544027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846e7ee-2cb3-4f6e-b72d-3f70581f1d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:28.544738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846e7ee-2cb3-4f6e-b72d-3f70581f1d19 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:28.545024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846e7ee-2cb3-4f6e-b72d-3f70581f1d19 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:28.545300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846e7ee-2cb3-4f6e-b72d-3f70581f1d19 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:28.546066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846e7ee-2cb3-4f6e-b72d-3f70581f1d19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:28.551227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846e7ee-2cb3-4f6e-b72d-3f70581f1d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:28.552727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846e7ee-2cb3-4f6e-b72d-3f70581f1d19 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:28.563281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae91477-9137-4459-8310-86c3f590c7cc None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:28.567663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 118/230] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:28.573899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846e7ee-2cb3-4f6e-b72d-3f70581f1d19 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:28.612034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4f8f1-b1db-4ae3-9c0a-021e4f47a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:28.613514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4f8f1-b1db-4ae3-9c0a-021e4f47a228 None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:28.613842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4f8f1-b1db-4ae3-9c0a-021e4f47a228 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:28.614124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4f8f1-b1db-4ae3-9c0a-021e4f47a228 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de/users/159fd9f414d1444ca9249426decfb3b1/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:28.621588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4f8f1-b1db-4ae3-9c0a-021e4f47a228 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b25ac538fb448279a63db11f84f70de, user_id=159fd9f414d1444ca9249426decfb3b1, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:28.623063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca4f8f1-b1db-4ae3-9c0a-021e4f47a228 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:28.655092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 113/231] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:35:28 2022] PUT /identity/v3/projects/9b25ac538fb448279a63db11f84f70de/users/159fd9f414d1444ca9249426decfb3b1/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:35:28.729626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846e7ee-2cb3-4f6e-b72d-3f70581f1d19 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:28.777340 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc648a1-e202-4b5a-a56d-3f462fa194a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:28.778156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc648a1-e202-4b5a-a56d-3f462fa194a5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:28.778439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc648a1-e202-4b5a-a56d-3f462fa194a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:28.778714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc648a1-e202-4b5a-a56d-3f462fa194a5 None admin] PATH_INFO: `/v3/users/159fd9f414d1444ca9249426decfb3b1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:28.780018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc648a1-e202-4b5a-a56d-3f462fa194a5 None admin] RBAC: Authorizing `identity:get_user(user_id=159fd9f414d1444ca9249426decfb3b1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:28.781508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc648a1-e202-4b5a-a56d-3f462fa194a5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:28.783736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 119/232] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:28 2022] GET /identity/v3/users/159fd9f414d1444ca9249426decfb3b1 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:28.789104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4f8f1-b1db-4ae3-9c0a-021e4f47a228 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:28.932843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4f8f1-b1db-4ae3-9c0a-021e4f47a228 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:28.973983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:28.981229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001b8b7-e3e3-4889-8f5a-44f40239fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:28.982755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001b8b7-e3e3-4889-8f5a-44f40239fabb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:28.983283 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001b8b7-e3e3-4889-8f5a-44f40239fabb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:28.983621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001b8b7-e3e3-4889-8f5a-44f40239fabb None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:28.985102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001b8b7-e3e3-4889-8f5a-44f40239fabb None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:28.988055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c001b8b7-e3e3-4889-8f5a-44f40239fabb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:28.992897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 114/233] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:28 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:28.995630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc648a1-e202-4b5a-a56d-3f462fa194a5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:29.102259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:29.156872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc648a1-e202-4b5a-a56d-3f462fa194a5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:29.207300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8cb5b0-e042-4679-8cc1-e41c6b1247ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:29.208202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cb5b0-e042-4679-8cc1-e41c6b1247ed None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:29.208482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cb5b0-e042-4679-8cc1-e41c6b1247ed None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:29.208789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8cb5b0-e042-4679-8cc1-e41c6b1247ed None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de/users/159fd9f414d1444ca9249426decfb3b1/roles/32433ed911104bb68970c0a00d0ddd49` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:29.211072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cb5b0-e042-4679-8cc1-e41c6b1247ed None admin] RBAC: Authorizing `identity:create_grant(project_id=9b25ac538fb448279a63db11f84f70de, user_id=159fd9f414d1444ca9249426decfb3b1, role_id=32433ed911104bb68970c0a00d0ddd49)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:29.212533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8cb5b0-e042-4679-8cc1-e41c6b1247ed None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:29.240943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 120/234] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:35:28 2022] PUT /identity/v3/projects/9b25ac538fb448279a63db11f84f70de/users/159fd9f414d1444ca9249426decfb3b1/roles/32433ed911104bb68970c0a00d0ddd49 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:29.397312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c27de7-8de4-4447-9969-094f5e2b4449 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:29.397312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c27de7-8de4-4447-9969-094f5e2b4449 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:29.397312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c27de7-8de4-4447-9969-094f5e2b4449 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:29.399168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 115/235] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:29.404667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ebc38-8123-42f6-a329-cd905e768896 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:29.405085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ebc38-8123-42f6-a329-cd905e768896 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:29.405266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ebc38-8123-42f6-a329-cd905e768896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:29.426282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0c5ebc38-8123-42f6-a329-cd905e768896 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:29.427471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0c5ebc38-8123-42f6-a329-cd905e768896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:29.434594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b74d-0b03-4436-b6ef-abc89dfc54cc None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:29.435179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b74d-0b03-4436-b6ef-abc89dfc54cc None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:29.435681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9b74d-0b03-4436-b6ef-abc89dfc54cc None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:29.437307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 116/236] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:29.442747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4852676-faff-4ced-b7a4-e25f8b9c9c50 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:29.443029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4852676-faff-4ced-b7a4-e25f8b9c9c50 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:29.444046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4852676-faff-4ced-b7a4-e25f8b9c9c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:29.474162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d4852676-faff-4ced-b7a4-e25f8b9c9c50 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:29.477365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d4852676-faff-4ced-b7a4-e25f8b9c9c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:29.543525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 121/237] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:29.572322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 117/238] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:29.578477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ebc38-8123-42f6-a329-cd905e768896 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:29.612888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4852676-faff-4ced-b7a4-e25f8b9c9c50 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:29.659167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:29.764909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ebc38-8123-42f6-a329-cd905e768896 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:29.780335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4852676-faff-4ced-b7a4-e25f8b9c9c50 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:29.820955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:29.836450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2cba3-4520-4317-8dac-44d6ab9acbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:29.837289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2cba3-4520-4317-8dac-44d6ab9acbce None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:29.837572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2cba3-4520-4317-8dac-44d6ab9acbce None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:29.837849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2cba3-4520-4317-8dac-44d6ab9acbce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:29.840348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbfd62-1752-43ca-9b0f-d7e20b6a1a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:29.841169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbfd62-1752-43ca-9b0f-d7e20b6a1a88 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:29.841445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbfd62-1752-43ca-9b0f-d7e20b6a1a88 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:29.841718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dbfd62-1752-43ca-9b0f-d7e20b6a1a88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:29.843976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dbfd62-1752-43ca-9b0f-d7e20b6a1a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:29.845992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dbfd62-1752-43ca-9b0f-d7e20b6a1a88 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:29.848211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 122/239] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:29.856605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbfd62-1752-43ca-9b0f-d7e20b6a1a88 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:29.863031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2cba3-4520-4317-8dac-44d6ab9acbce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-1cb2cba3-4520-4317-8dac-44d6ab9acbce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:29.872768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:29.873642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:29.874393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:29.877472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 118/240] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:29.883017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2cba3-4520-4317-8dac-44d6ab9acbce None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:29.965008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dbfd62-1752-43ca-9b0f-d7e20b6a1a88 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:29.977491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:30.004617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd214694-d5ed-4845-ba1c-f8bacd5fc797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:30.004617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214694-d5ed-4845-ba1c-f8bacd5fc797 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.004617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214694-d5ed-4845-ba1c-f8bacd5fc797 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.004617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214694-d5ed-4845-ba1c-f8bacd5fc797 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.005033 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd214694-d5ed-4845-ba1c-f8bacd5fc797 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:30.005033 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd214694-d5ed-4845-ba1c-f8bacd5fc797 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:30.039024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 123/241] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:35:29 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:35:30.089102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2cba3-4520-4317-8dac-44d6ab9acbce None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:30.136472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e84c2-9a97-4a54-b128-21f31778142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:30.137179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e84c2-9a97-4a54-b128-21f31778142c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.137467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e84c2-9a97-4a54-b128-21f31778142c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.137748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e84c2-9a97-4a54-b128-21f31778142c None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.138505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e84c2-9a97-4a54-b128-21f31778142c None admin] RBAC: Adding query filter params (name=reader) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:30.138816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e84c2-9a97-4a54-b128-21f31778142c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:30.140201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e84c2-9a97-4a54-b128-21f31778142c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:30.151177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 119/242] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:30.151546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa979a9-335c-49bf-bc26-2b3cda50c019 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.151841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa979a9-335c-49bf-bc26-2b3cda50c019 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.152122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa979a9-335c-49bf-bc26-2b3cda50c019 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.154756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 124/243] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:30.156796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e84c2-9a97-4a54-b128-21f31778142c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:30.160335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f040d-5207-4ab5-a2a2-78f4c6128205 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.160698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f040d-5207-4ab5-a2a2-78f4c6128205 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.160986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f040d-5207-4ab5-a2a2-78f4c6128205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.196634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-fd1f040d-5207-4ab5-a2a2-78f4c6128205 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:30.196634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-fd1f040d-5207-4ab5-a2a2-78f4c6128205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:30.242462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:30.299843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 125/244] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:30.324159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f040d-5207-4ab5-a2a2-78f4c6128205 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:30.332272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e84c2-9a97-4a54-b128-21f31778142c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:30.376924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72287cfb-385d-4628-9463-cc2357798818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:30.378315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72287cfb-385d-4628-9463-cc2357798818 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.378828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72287cfb-385d-4628-9463-cc2357798818 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.379303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72287cfb-385d-4628-9463-cc2357798818 None admin] PATH_INFO: `/v3/users/c0b5fd91f511465282a9f388ee1eadef` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.395499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72287cfb-385d-4628-9463-cc2357798818 None admin] RBAC: Authorizing `identity:get_user(user_id=c0b5fd91f511465282a9f388ee1eadef)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:30.397648 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72287cfb-385d-4628-9463-cc2357798818 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:30.406505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 120/245] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:30 2022] GET /identity/v3/users/c0b5fd91f511465282a9f388ee1eadef => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:30.407848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72287cfb-385d-4628-9463-cc2357798818 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:30.445043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f040d-5207-4ab5-a2a2-78f4c6128205 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:30.452930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:30.493496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263f13-3354-4096-ad3c-c308efb47b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:30.494223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263f13-3354-4096-ad3c-c308efb47b36 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.494504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263f13-3354-4096-ad3c-c308efb47b36 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.494779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01263f13-3354-4096-ad3c-c308efb47b36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.511425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01263f13-3354-4096-ad3c-c308efb47b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-01263f13-3354-4096-ad3c-c308efb47b36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:30.519865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:30.520771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:30.521518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 126/246] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:30.527452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72287cfb-385d-4628-9463-cc2357798818 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:30.529553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263f13-3354-4096-ad3c-c308efb47b36 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:30.540543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:30.584439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7fe8f-2f48-423e-9c31-a75a3a69c658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:30.585135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7fe8f-2f48-423e-9c31-a75a3a69c658 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.585467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7fe8f-2f48-423e-9c31-a75a3a69c658 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.585741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7fe8f-2f48-423e-9c31-a75a3a69c658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.667423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7fe8f-2f48-423e-9c31-a75a3a69c658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:30.668930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7fe8f-2f48-423e-9c31-a75a3a69c658 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:30.672024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01263f13-3354-4096-ad3c-c308efb47b36 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:30.717769 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa37a0-395c-472a-b802-42a36aefdbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:30.718557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa37a0-395c-472a-b802-42a36aefdbd9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.718844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa37a0-395c-472a-b802-42a36aefdbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.719219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa37a0-395c-472a-b802-42a36aefdbd9 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.720029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa37a0-395c-472a-b802-42a36aefdbd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:30.721031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa37a0-395c-472a-b802-42a36aefdbd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:30.722464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa37a0-395c-472a-b802-42a36aefdbd9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:30.732320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 127/247] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:30.739002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa37a0-395c-472a-b802-42a36aefdbd9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:30.769164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:30.790544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 121/248] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:30.796399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7fe8f-2f48-423e-9c31-a75a3a69c658 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:30.808043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:30.863054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa37a0-395c-472a-b802-42a36aefdbd9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:30.903275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d602bb-4e38-41cf-881e-51b8eb6743d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:30.904264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d602bb-4e38-41cf-881e-51b8eb6743d1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.904549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d602bb-4e38-41cf-881e-51b8eb6743d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.904849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d602bb-4e38-41cf-881e-51b8eb6743d1 None admin] PATH_INFO: `/v3/users/e45c1ccf481845c7a84a4d2f4113a84c` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.906066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d602bb-4e38-41cf-881e-51b8eb6743d1 None admin] RBAC: Authorizing `identity:get_user(user_id=e45c1ccf481845c7a84a4d2f4113a84c)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:30.906870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7fe8f-2f48-423e-9c31-a75a3a69c658 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:30.907835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d602bb-4e38-41cf-881e-51b8eb6743d1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:30.910454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 128/249] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:30 2022] GET /identity/v3/users/e45c1ccf481845c7a84a4d2f4113a84c => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:30.915828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d602bb-4e38-41cf-881e-51b8eb6743d1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:30.943742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e530d9-d05a-4fcf-99c3-6c85f4696993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:30.944631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e530d9-d05a-4fcf-99c3-6c85f4696993 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:30.944919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e530d9-d05a-4fcf-99c3-6c85f4696993 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:30.945192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e530d9-d05a-4fcf-99c3-6c85f4696993 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:30.946402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e530d9-d05a-4fcf-99c3-6c85f4696993 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:30.947843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e530d9-d05a-4fcf-99c3-6c85f4696993 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:30.949573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 122/250] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:30 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:30.958228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e530d9-d05a-4fcf-99c3-6c85f4696993 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:31.030257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:31.046650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d602bb-4e38-41cf-881e-51b8eb6743d1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:31.060028 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:31.086252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e530d9-d05a-4fcf-99c3-6c85f4696993 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:31.092128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91741d5-68e8-4e43-8ec0-f1bf26088959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:31.093055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91741d5-68e8-4e43-8ec0-f1bf26088959 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:31.093342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91741d5-68e8-4e43-8ec0-f1bf26088959 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:31.093620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91741d5-68e8-4e43-8ec0-f1bf26088959 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:31.094995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91741d5-68e8-4e43-8ec0-f1bf26088959 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:31.097309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91741d5-68e8-4e43-8ec0-f1bf26088959 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:31.099456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 129/251] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:30 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:31.104893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91741d5-68e8-4e43-8ec0-f1bf26088959 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:31.122410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d45a0-1118-4b61-897c-470fabbe7bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:31.123114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d45a0-1118-4b61-897c-470fabbe7bbd None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:31.123403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d45a0-1118-4b61-897c-470fabbe7bbd None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:31.123682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d45a0-1118-4b61-897c-470fabbe7bbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:31.124518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d45a0-1118-4b61-897c-470fabbe7bbd None admin] RBAC: Adding query filter params (role.id=3f15101d95de479cacc1f6d91a2ff504, scope.project.id=9b25ac538fb448279a63db11f84f70de, user.id=c0b5fd91f511465282a9f388ee1eadef) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:31.124903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d45a0-1118-4b61-897c-470fabbe7bbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:31.126873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d45a0-1118-4b61-897c-470fabbe7bbd None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:31.133085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 123/252] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:30 2022] GET /identity/v3/role_assignments?user.id=c0b5fd91f511465282a9f388ee1eadef&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=3f15101d95de479cacc1f6d91a2ff504 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:31.259198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91741d5-68e8-4e43-8ec0-f1bf26088959 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:31.299902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf28b3a-38a4-4a52-89eb-2a3b7ff5a592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:31.300767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf28b3a-38a4-4a52-89eb-2a3b7ff5a592 None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:31.301053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf28b3a-38a4-4a52-89eb-2a3b7ff5a592 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:31.301330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf28b3a-38a4-4a52-89eb-2a3b7ff5a592 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de/users/e45c1ccf481845c7a84a4d2f4113a84c/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:31.303357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf28b3a-38a4-4a52-89eb-2a3b7ff5a592 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b25ac538fb448279a63db11f84f70de, user_id=e45c1ccf481845c7a84a4d2f4113a84c, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:31.304835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf28b3a-38a4-4a52-89eb-2a3b7ff5a592 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:31.337963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 130/253] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:35:31 2022] PUT /identity/v3/projects/9b25ac538fb448279a63db11f84f70de/users/e45c1ccf481845c7a84a4d2f4113a84c/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:31.891916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e086f-9189-4f64-badc-d7bb7d9a39a6 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:31.892646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e086f-9189-4f64-badc-d7bb7d9a39a6 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:31.892646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e086f-9189-4f64-badc-d7bb7d9a39a6 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:31.893985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 124/254] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:31.899597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfebeec-dcaa-4c4e-a592-32fe3e7a1187 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:31.900074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfebeec-dcaa-4c4e-a592-32fe3e7a1187 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:31.900242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfebeec-dcaa-4c4e-a592-32fe3e7a1187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:31.922292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-2dfebeec-dcaa-4c4e-a592-32fe3e7a1187 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:31.924248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2dfebeec-dcaa-4c4e-a592-32fe3e7a1187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:31.943479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:31.988556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb27d1-462e-4762-96bc-88e786994ee1 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:31.988914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb27d1-462e-4762-96bc-88e786994ee1 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:31.989143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb27d1-462e-4762-96bc-88e786994ee1 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:31.991582 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 125/255] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:31.998059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8286e-e5bc-46eb-bee5-ccaec3efa97b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:31.998349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8286e-e5bc-46eb-bee5-ccaec3efa97b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:31.998631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8286e-e5bc-46eb-bee5-ccaec3efa97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:32.024233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 131/256] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:32.025201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a8d8286e-e5bc-46eb-bee5-ccaec3efa97b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:32.026409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a8d8286e-e5bc-46eb-bee5-ccaec3efa97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:32.060285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfebeec-dcaa-4c4e-a592-32fe3e7a1187 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:32.144078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 126/257] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:32.197406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8286e-e5bc-46eb-bee5-ccaec3efa97b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:32.252739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfebeec-dcaa-4c4e-a592-32fe3e7a1187 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:32.322297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83901f4-4784-4654-a0de-0042eafac490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:32.323107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83901f4-4784-4654-a0de-0042eafac490 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:32.323396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83901f4-4784-4654-a0de-0042eafac490 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:32.323671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83901f4-4784-4654-a0de-0042eafac490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:32.342504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:32.342675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83901f4-4784-4654-a0de-0042eafac490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-f83901f4-4784-4654-a0de-0042eafac490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:32.359922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:32.362050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:32.362830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 132/258] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:32.370663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83901f4-4784-4654-a0de-0042eafac490 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:32.402715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:32.421163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8286e-e5bc-46eb-bee5-ccaec3efa97b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:32.514664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf42a6b-2967-458a-bf2f-ff070895a66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:32.515491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf42a6b-2967-458a-bf2f-ff070895a66c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:32.515779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf42a6b-2967-458a-bf2f-ff070895a66c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:32.516060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf42a6b-2967-458a-bf2f-ff070895a66c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:32.548724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf42a6b-2967-458a-bf2f-ff070895a66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-bcf42a6b-2967-458a-bf2f-ff070895a66c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:32.564874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:32.565780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:32.566544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 127/259] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:32.571981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf42a6b-2967-458a-bf2f-ff070895a66c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:32.621107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83901f4-4784-4654-a0de-0042eafac490 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:32.691365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b8acd-fc3e-4806-aa48-081ce07a5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:32.691969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b8acd-fc3e-4806-aa48-081ce07a5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:32.692250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b8acd-fc3e-4806-aa48-081ce07a5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:32.692523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b8acd-fc3e-4806-aa48-081ce07a5ef2 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:32.693317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b8acd-fc3e-4806-aa48-081ce07a5ef2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:32.693624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b8acd-fc3e-4806-aa48-081ce07a5ef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:32.695017 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b8acd-fc3e-4806-aa48-081ce07a5ef2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:32.716284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 133/260] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:32.721855 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b8acd-fc3e-4806-aa48-081ce07a5ef2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:32.819154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf42a6b-2967-458a-bf2f-ff070895a66c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:32.873942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:32.895067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:32.921290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ab0a9-bbf4-4516-8972-9a9ee7ac2ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:32.921971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ab0a9-bbf4-4516-8972-9a9ee7ac2ab4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:32.922254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ab0a9-bbf4-4516-8972-9a9ee7ac2ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:32.922534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ab0a9-bbf4-4516-8972-9a9ee7ac2ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:32.924310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ab0a9-bbf4-4516-8972-9a9ee7ac2ab4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:32.924668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ab0a9-bbf4-4516-8972-9a9ee7ac2ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:32.926096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ab0a9-bbf4-4516-8972-9a9ee7ac2ab4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:32.944108 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 128/261] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:32.949433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ab0a9-bbf4-4516-8972-9a9ee7ac2ab4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:32.952172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b8acd-fc3e-4806-aa48-081ce07a5ef2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:33.049938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62964312-f799-46a8-b803-dcb589a25762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:33.051216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62964312-f799-46a8-b803-dcb589a25762 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.051695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62964312-f799-46a8-b803-dcb589a25762 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.052185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62964312-f799-46a8-b803-dcb589a25762 None admin] PATH_INFO: `/v3/users/159fd9f414d1444ca9249426decfb3b1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.054130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62964312-f799-46a8-b803-dcb589a25762 None admin] RBAC: Authorizing `identity:get_user(user_id=159fd9f414d1444ca9249426decfb3b1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:33.055775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62964312-f799-46a8-b803-dcb589a25762 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:33.057970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 134/262] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:32 2022] GET /identity/v3/users/159fd9f414d1444ca9249426decfb3b1 => generated 283 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:33.065431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62964312-f799-46a8-b803-dcb589a25762 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:33.185921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62964312-f799-46a8-b803-dcb589a25762 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:33.207054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ab0a9-bbf4-4516-8972-9a9ee7ac2ab4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:33.225904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f577d7-483c-4501-9fcb-5a46335b9e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:33.227014 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f577d7-483c-4501-9fcb-5a46335b9e90 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.227493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f577d7-483c-4501-9fcb-5a46335b9e90 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.227978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f577d7-483c-4501-9fcb-5a46335b9e90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:33.244014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75491d4b-f838-4360-97bc-144d242d0f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:33.244931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75491d4b-f838-4360-97bc-144d242d0f84 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.245210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75491d4b-f838-4360-97bc-144d242d0f84 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.245480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75491d4b-f838-4360-97bc-144d242d0f84 None admin] PATH_INFO: `/v3/users/159fd9f414d1444ca9249426decfb3b1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.246717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75491d4b-f838-4360-97bc-144d242d0f84 None admin] RBAC: Authorizing `identity:get_user(user_id=159fd9f414d1444ca9249426decfb3b1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:33.248176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75491d4b-f838-4360-97bc-144d242d0f84 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:33.250365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 129/263] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:32 2022] GET /identity/v3/users/159fd9f414d1444ca9249426decfb3b1 => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:33.256147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75491d4b-f838-4360-97bc-144d242d0f84 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:33.314542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f577d7-483c-4501-9fcb-5a46335b9e90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:33.315758 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:33.317769 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f577d7-483c-4501-9fcb-5a46335b9e90 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:33.385110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75491d4b-f838-4360-97bc-144d242d0f84 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:33.430328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 135/264] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:33.435673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f577d7-483c-4501-9fcb-5a46335b9e90 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:33.437416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a40fe-74f9-478f-80cf-e8d7db086b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:33.438281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a40fe-74f9-478f-80cf-e8d7db086b73 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.439302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a40fe-74f9-478f-80cf-e8d7db086b73 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.439758 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a40fe-74f9-478f-80cf-e8d7db086b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.506480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:33.511210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a40fe-74f9-478f-80cf-e8d7db086b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:33.512960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a40fe-74f9-478f-80cf-e8d7db086b73 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:33.573312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:33.580612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f577d7-483c-4501-9fcb-5a46335b9e90 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:33.618986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 130/265] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 23:35:33.620666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551248-35ed-4d05-be77-def26a2d2b3b None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.621149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551248-35ed-4d05-be77-def26a2d2b3b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.621653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1551248-35ed-4d05-be77-def26a2d2b3b None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.623224 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 131/266] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:33.628297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4151190-6bf2-41e7-9b20-cb5bd3b2b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:33.628297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4151190-6bf2-41e7-9b20-cb5bd3b2b6cc None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.628297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4151190-6bf2-41e7-9b20-cb5bd3b2b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.628297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4151190-6bf2-41e7-9b20-cb5bd3b2b6cc None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.629641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1551248-35ed-4d05-be77-def26a2d2b3b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:33.634288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4151190-6bf2-41e7-9b20-cb5bd3b2b6cc None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:33.635043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4151190-6bf2-41e7-9b20-cb5bd3b2b6cc None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:33.636991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 136/267] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:33 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:33.638036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bd25d-fa4c-418b-988a-210705529f82 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.638323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bd25d-fa4c-418b-988a-210705529f82 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.638603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bd25d-fa4c-418b-988a-210705529f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.667613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-244bd25d-fa4c-418b-988a-210705529f82 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:33.668959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-244bd25d-fa4c-418b-988a-210705529f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:33.764008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1551248-35ed-4d05-be77-def26a2d2b3b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:33.764941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 137/268] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:33.765818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bd25d-fa4c-418b-988a-210705529f82 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:33.802093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27985be-e9a5-437b-afe4-9f71203ab829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:33.803334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27985be-e9a5-437b-afe4-9f71203ab829 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.803806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27985be-e9a5-437b-afe4-9f71203ab829 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.804267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27985be-e9a5-437b-afe4-9f71203ab829 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.805772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27985be-e9a5-437b-afe4-9f71203ab829 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:33.807420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27985be-e9a5-437b-afe4-9f71203ab829 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:33.809369 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 132/269] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:33 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:33.810331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27985be-e9a5-437b-afe4-9f71203ab829 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:33.831088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:33.844464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:33.918467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bd25d-fa4c-418b-988a-210705529f82 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:33.921128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27985be-e9a5-437b-afe4-9f71203ab829 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:33.959677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed718b6-a054-4b4d-9982-44850af6422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:33.964604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed718b6-a054-4b4d-9982-44850af6422f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.964604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed718b6-a054-4b4d-9982-44850af6422f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.964604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed718b6-a054-4b4d-9982-44850af6422f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.979591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed718b6-a054-4b4d-9982-44850af6422f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:33.980891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f178d-824f-47ab-b5a0-aef9e204ef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:33.982093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f178d-824f-47ab-b5a0-aef9e204ef4e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.982541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f178d-824f-47ab-b5a0-aef9e204ef4e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:33.982982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f178d-824f-47ab-b5a0-aef9e204ef4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:33.984009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f178d-824f-47ab-b5a0-aef9e204ef4e None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=9b25ac538fb448279a63db11f84f70de, user.id=159fd9f414d1444ca9249426decfb3b1) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:33.984488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f178d-824f-47ab-b5a0-aef9e204ef4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:33.986770 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f178d-824f-47ab-b5a0-aef9e204ef4e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-6ed718b6-a054-4b4d-9982-44850af6422f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:33.989120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:33.989958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:33.990708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 133/270] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:33.993344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed718b6-a054-4b4d-9982-44850af6422f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:33.997261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 138/271] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:33 2022] GET /identity/v3/role_assignments?user.id=159fd9f414d1444ca9249426decfb3b1&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:33.998815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d32df3-2270-42e7-9c4c-40a89729d2b3 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:33.999622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d32df3-2270-42e7-9c4c-40a89729d2b3 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.001282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d32df3-2270-42e7-9c4c-40a89729d2b3 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.003269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 139/272] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:34.004330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d32df3-2270-42e7-9c4c-40a89729d2b3 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:34.176545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:34.192450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed718b6-a054-4b4d-9982-44850af6422f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:34.213996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d32df3-2270-42e7-9c4c-40a89729d2b3 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:34.216207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:34.239971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdab18-23c0-430c-aeca-5ec1912fd309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:34.240654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdab18-23c0-430c-aeca-5ec1912fd309 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.241099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdab18-23c0-430c-aeca-5ec1912fd309 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.241634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdab18-23c0-430c-aeca-5ec1912fd309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.242474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdab18-23c0-430c-aeca-5ec1912fd309 None admin] RBAC: Adding query filter params (role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=9b25ac538fb448279a63db11f84f70de, user.id=159fd9f414d1444ca9249426decfb3b1) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:34.243017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdab18-23c0-430c-aeca-5ec1912fd309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:34.244701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdab18-23c0-430c-aeca-5ec1912fd309 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:34.252540 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 134/273] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:33 2022] GET /identity/v3/role_assignments?user.id=159fd9f414d1444ca9249426decfb3b1&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:34.254274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a574fde-dc8b-4cf3-9a99-90af2649bf74 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.254783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a574fde-dc8b-4cf3-9a99-90af2649bf74 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.255253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a574fde-dc8b-4cf3-9a99-90af2649bf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.258022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924c859-8cfc-4aab-9496-fe989537f588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:34.260956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924c859-8cfc-4aab-9496-fe989537f588 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.260956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924c859-8cfc-4aab-9496-fe989537f588 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.260956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924c859-8cfc-4aab-9496-fe989537f588 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.260956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924c859-8cfc-4aab-9496-fe989537f588 None admin] RBAC: Adding query filter params (name=member) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:34.260956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924c859-8cfc-4aab-9496-fe989537f588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:34.261881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924c859-8cfc-4aab-9496-fe989537f588 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:34.278077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 140/274] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:34.278077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b966d-fea8-4923-acaa-e8947796847e None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.278077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b966d-fea8-4923-acaa-e8947796847e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.278077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b966d-fea8-4923-acaa-e8947796847e None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.278453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 141/275] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:34.282802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0a574fde-dc8b-4cf3-9a99-90af2649bf74 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:34.284103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b966d-fea8-4923-acaa-e8947796847e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:34.285029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0a574fde-dc8b-4cf3-9a99-90af2649bf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:34.384743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 135/276] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:34.384743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd435e-50af-408f-93ca-2f4790064956 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.385515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd435e-50af-408f-93ca-2f4790064956 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.385809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd435e-50af-408f-93ca-2f4790064956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.409893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-46cd435e-50af-408f-93ca-2f4790064956 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:34.411172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-46cd435e-50af-408f-93ca-2f4790064956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:34.450936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b966d-fea8-4923-acaa-e8947796847e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:34.490176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e292fd-a14e-4ebb-9830-bd32b7d1295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:34.491066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e292fd-a14e-4ebb-9830-bd32b7d1295a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.491350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e292fd-a14e-4ebb-9830-bd32b7d1295a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.491625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e292fd-a14e-4ebb-9830-bd32b7d1295a None admin] PATH_INFO: `/v3/users/e5013b666a824564b7b9b1009c09a44b` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.499770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 136/277] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:34.501179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd435e-50af-408f-93ca-2f4790064956 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:34.507160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e292fd-a14e-4ebb-9830-bd32b7d1295a None admin] RBAC: Authorizing `identity:get_user(user_id=e5013b666a824564b7b9b1009c09a44b)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:34.508948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e292fd-a14e-4ebb-9830-bd32b7d1295a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:34.510862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 142/278] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:34 2022] GET /identity/v3/users/e5013b666a824564b7b9b1009c09a44b => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:34.515472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fbdc0-e2ea-4ab0-8954-9b5b478e7948 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.515472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fbdc0-e2ea-4ab0-8954-9b5b478e7948 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.515472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fbdc0-e2ea-4ab0-8954-9b5b478e7948 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.517823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:34.518660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 143/279] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:34.519558 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fbdc0-e2ea-4ab0-8954-9b5b478e7948 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:34.545638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:34.615834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd435e-50af-408f-93ca-2f4790064956 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:34.625541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fbdc0-e2ea-4ab0-8954-9b5b478e7948 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:34.649661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cda962-ca85-45aa-a874-7fcd37b4430f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:34.650456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cda962-ca85-45aa-a874-7fcd37b4430f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.650742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cda962-ca85-45aa-a874-7fcd37b4430f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.651022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cda962-ca85-45aa-a874-7fcd37b4430f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.652192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cda962-ca85-45aa-a874-7fcd37b4430f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:34.653691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cda962-ca85-45aa-a874-7fcd37b4430f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:34.655318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 137/280] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:34.656081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cda962-ca85-45aa-a874-7fcd37b4430f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:34.662110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0fc36-c272-42ec-89d0-5bda6663593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:34.662773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0fc36-c272-42ec-89d0-5bda6663593a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.663054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0fc36-c272-42ec-89d0-5bda6663593a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.663330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0fc36-c272-42ec-89d0-5bda6663593a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.664159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0fc36-c272-42ec-89d0-5bda6663593a None admin] RBAC: Adding query filter params (role.id=86c49ec8587244e48fab81862ddb3641, user.id=e5013b666a824564b7b9b1009c09a44b, scope.system=all) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:34.664468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0fc36-c272-42ec-89d0-5bda6663593a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:34.666315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0fc36-c272-42ec-89d0-5bda6663593a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:34.672618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 144/281] 10.222.0.47 () {56 vars in 1308 bytes} [Fri Jun 24 23:35:34 2022] GET /identity/v3/role_assignments?user.id=e5013b666a824564b7b9b1009c09a44b&scope.system=all&role.id=86c49ec8587244e48fab81862ddb3641 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:34.672778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f679dd-7796-410e-ba8f-945f551356a7 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.673016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f679dd-7796-410e-ba8f-945f551356a7 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.673506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f679dd-7796-410e-ba8f-945f551356a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.691220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c8f679dd-7796-410e-ba8f-945f551356a7 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:34.692450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c8f679dd-7796-410e-ba8f-945f551356a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:34.758338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:34.777739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 145/282] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:34.781587 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f679dd-7796-410e-ba8f-945f551356a7 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:34.785864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cda962-ca85-45aa-a874-7fcd37b4430f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:34.815712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:34.829036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bb606-b621-4525-9990-7ec16891148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:34.829883 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bb606-b621-4525-9990-7ec16891148e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.830351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bb606-b621-4525-9990-7ec16891148e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.831183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bb606-b621-4525-9990-7ec16891148e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.841222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32bb606-b621-4525-9990-7ec16891148e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-e32bb606-b621-4525-9990-7ec16891148e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:34.849812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:34.850666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:34.851735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 138/283] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:34.856825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bb606-b621-4525-9990-7ec16891148e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:34.913178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f679dd-7796-410e-ba8f-945f551356a7 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:34.953631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a06d1-daab-47cb-b409-00251343fec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:34.954550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a06d1-daab-47cb-b409-00251343fec2 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:34.954833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a06d1-daab-47cb-b409-00251343fec2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:34.955109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a06d1-daab-47cb-b409-00251343fec2 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:34.955824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a06d1-daab-47cb-b409-00251343fec2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:34.958820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a06d1-daab-47cb-b409-00251343fec2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:34.969457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bb606-b621-4525-9990-7ec16891148e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:34.983642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 146/284] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:35:34 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:35:34.984417 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a06d1-daab-47cb-b409-00251343fec2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:35.025766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893f35e-bd97-4597-a15f-c572a6f9cb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:35.027124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893f35e-bd97-4597-a15f-c572a6f9cb96 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:35.027591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893f35e-bd97-4597-a15f-c572a6f9cb96 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:35.029023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893f35e-bd97-4597-a15f-c572a6f9cb96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:35.040786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:35.047123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893f35e-bd97-4597-a15f-c572a6f9cb96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-b893f35e-bd97-4597-a15f-c572a6f9cb96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:35.059693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:35.060741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:35.061531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:35.065323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 139/285] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:35.071556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893f35e-bd97-4597-a15f-c572a6f9cb96 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:35.075131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:35.117698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a06d1-daab-47cb-b409-00251343fec2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:35.165486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faca149-2911-4762-890b-81d45a64cce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:35.166398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faca149-2911-4762-890b-81d45a64cce8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:35.166827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faca149-2911-4762-890b-81d45a64cce8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:35.167113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faca149-2911-4762-890b-81d45a64cce8 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:35.167890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faca149-2911-4762-890b-81d45a64cce8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:35.168204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faca149-2911-4762-890b-81d45a64cce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:35.170653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faca149-2911-4762-890b-81d45a64cce8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:35.184486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 147/286] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:35.190779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faca149-2911-4762-890b-81d45a64cce8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:35.203336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893f35e-bd97-4597-a15f-c572a6f9cb96 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:35.252722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81102e-d318-4697-8c55-2115106f718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:35.254244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81102e-d318-4697-8c55-2115106f718c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:35.254677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81102e-d318-4697-8c55-2115106f718c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:35.255357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81102e-d318-4697-8c55-2115106f718c None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:35.256638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81102e-d318-4697-8c55-2115106f718c None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:35.257093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81102e-d318-4697-8c55-2115106f718c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:35.260147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81102e-d318-4697-8c55-2115106f718c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:35.272264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 140/287] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:35.279381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81102e-d318-4697-8c55-2115106f718c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:35.315680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faca149-2911-4762-890b-81d45a64cce8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:35.357927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:35.363308 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c408df-a6d2-4f19-b29f-a57790a6a27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:35.364204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c408df-a6d2-4f19-b29f-a57790a6a27d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:35.364482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c408df-a6d2-4f19-b29f-a57790a6a27d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:35.364790 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c408df-a6d2-4f19-b29f-a57790a6a27d None admin] PATH_INFO: `/v3/users/c0b5fd91f511465282a9f388ee1eadef` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:35.366282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c408df-a6d2-4f19-b29f-a57790a6a27d None admin] RBAC: Authorizing `identity:get_user(user_id=c0b5fd91f511465282a9f388ee1eadef)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:35.367947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c408df-a6d2-4f19-b29f-a57790a6a27d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:35.371679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 148/288] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:35 2022] GET /identity/v3/users/c0b5fd91f511465282a9f388ee1eadef => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:35.377718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c408df-a6d2-4f19-b29f-a57790a6a27d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:35.381616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:35.406395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81102e-d318-4697-8c55-2115106f718c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:35.450510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51e14e-7018-4f05-b3a6-de8466132b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:35.451403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51e14e-7018-4f05-b3a6-de8466132b7f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:35.451691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51e14e-7018-4f05-b3a6-de8466132b7f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:35.452523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51e14e-7018-4f05-b3a6-de8466132b7f None admin] PATH_INFO: `/v3/users/e45c1ccf481845c7a84a4d2f4113a84c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:35.453824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51e14e-7018-4f05-b3a6-de8466132b7f None admin] RBAC: Authorizing `identity:get_user(user_id=e45c1ccf481845c7a84a4d2f4113a84c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:35.456724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51e14e-7018-4f05-b3a6-de8466132b7f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:35.459150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 141/289] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:35 2022] GET /identity/v3/users/e45c1ccf481845c7a84a4d2f4113a84c => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:35.465016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51e14e-7018-4f05-b3a6-de8466132b7f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:35.514419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c408df-a6d2-4f19-b29f-a57790a6a27d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:35.550520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4718d-3123-40fd-991b-c32e37993233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:35.551293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4718d-3123-40fd-991b-c32e37993233 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:35.551577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4718d-3123-40fd-991b-c32e37993233 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:35.551853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4718d-3123-40fd-991b-c32e37993233 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:35.553096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4718d-3123-40fd-991b-c32e37993233 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:35.554533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4718d-3123-40fd-991b-c32e37993233 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:35.556557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 149/290] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:35 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:35.562639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4718d-3123-40fd-991b-c32e37993233 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:35.587727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51e14e-7018-4f05-b3a6-de8466132b7f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:35.623577 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:35.641542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8560a5-fb12-48a3-86bd-86fa0f5ddd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:35.642232 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8560a5-fb12-48a3-86bd-86fa0f5ddd94 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:35.642514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8560a5-fb12-48a3-86bd-86fa0f5ddd94 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:35.642780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8560a5-fb12-48a3-86bd-86fa0f5ddd94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:35.680499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:35.721034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4718d-3123-40fd-991b-c32e37993233 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:35.775922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8560a5-fb12-48a3-86bd-86fa0f5ddd94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:35.779360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8560a5-fb12-48a3-86bd-86fa0f5ddd94 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:35.797299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf1617-dc73-476f-b55a-dc33ebe95f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:35.798832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf1617-dc73-476f-b55a-dc33ebe95f63 None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:35.799125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf1617-dc73-476f-b55a-dc33ebe95f63 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:35.799405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf1617-dc73-476f-b55a-dc33ebe95f63 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de/users/c0b5fd91f511465282a9f388ee1eadef/roles/3f15101d95de479cacc1f6d91a2ff504` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:35.803281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf1617-dc73-476f-b55a-dc33ebe95f63 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b25ac538fb448279a63db11f84f70de, user_id=c0b5fd91f511465282a9f388ee1eadef, role_id=3f15101d95de479cacc1f6d91a2ff504)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:35.804703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf1617-dc73-476f-b55a-dc33ebe95f63 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:35.827768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 150/291] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:35:35 2022] PUT /identity/v3/projects/9b25ac538fb448279a63db11f84f70de/users/c0b5fd91f511465282a9f388ee1eadef/roles/3f15101d95de479cacc1f6d91a2ff504 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:35.910295 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 142/292] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:35.915886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf1617-dc73-476f-b55a-dc33ebe95f63 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:36.085397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf1617-dc73-476f-b55a-dc33ebe95f63 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:36.119621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:36.126909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59903df4-cc72-4ebe-9837-42010c56c84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:36.127786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59903df4-cc72-4ebe-9837-42010c56c84d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:36.128085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59903df4-cc72-4ebe-9837-42010c56c84d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:36.128376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59903df4-cc72-4ebe-9837-42010c56c84d None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:36.129797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59903df4-cc72-4ebe-9837-42010c56c84d None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:36.131240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59903df4-cc72-4ebe-9837-42010c56c84d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:36.133266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 151/293] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:35 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:36.139078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8560a5-fb12-48a3-86bd-86fa0f5ddd94 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:36.170964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:36.257486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8560a5-fb12-48a3-86bd-86fa0f5ddd94 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:36.307976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8396ca0-ff3a-462d-a393-66379c4535f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:36.308682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8396ca0-ff3a-462d-a393-66379c4535f2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:36.308972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8396ca0-ff3a-462d-a393-66379c4535f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:36.309246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8396ca0-ff3a-462d-a393-66379c4535f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:36.310080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8396ca0-ff3a-462d-a393-66379c4535f2 None admin] RBAC: Adding query filter params (role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=9b25ac538fb448279a63db11f84f70de, user.id=e45c1ccf481845c7a84a4d2f4113a84c) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:36.310388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8396ca0-ff3a-462d-a393-66379c4535f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:36.312173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8396ca0-ff3a-462d-a393-66379c4535f2 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:36.321871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 143/294] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:36 2022] GET /identity/v3/role_assignments?user.id=e45c1ccf481845c7a84a4d2f4113a84c&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=86c49ec8587244e48fab81862ddb3641 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:36.865428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fac1fe-3ed3-47ef-b900-56b985994d82 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:36.866123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fac1fe-3ed3-47ef-b900-56b985994d82 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:36.866666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fac1fe-3ed3-47ef-b900-56b985994d82 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:36.868291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 152/295] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:36.874070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb8a6a-2bc7-4d1f-916b-29d5606da5d9 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:36.874368 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb8a6a-2bc7-4d1f-916b-29d5606da5d9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:36.874656 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb8a6a-2bc7-4d1f-916b-29d5606da5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:36.892567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-4fcb8a6a-2bc7-4d1f-916b-29d5606da5d9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:36.893886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4fcb8a6a-2bc7-4d1f-916b-29d5606da5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:36.977704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 144/296] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:37.001566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fac1fe-3ed3-47ef-b900-56b985994d82 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:37.107479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fac1fe-3ed3-47ef-b900-56b985994d82 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:37.146981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba70c1-79f9-4a6c-8f6c-2e17c015cab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:37.147702 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba70c1-79f9-4a6c-8f6c-2e17c015cab6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:37.148240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba70c1-79f9-4a6c-8f6c-2e17c015cab6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:37.148514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba70c1-79f9-4a6c-8f6c-2e17c015cab6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:37.159877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba70c1-79f9-4a6c-8f6c-2e17c015cab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-8eba70c1-79f9-4a6c-8f6c-2e17c015cab6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:37.167568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:37.168424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:37.169284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 153/297] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:37.175572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb8a6a-2bc7-4d1f-916b-29d5606da5d9 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:37.201556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:37.284509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb8a6a-2bc7-4d1f-916b-29d5606da5d9 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:37.323145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6dcd4-be84-42ec-a55e-6e983e772380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:37.323838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6dcd4-be84-42ec-a55e-6e983e772380 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:37.324118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6dcd4-be84-42ec-a55e-6e983e772380 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:37.324399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6dcd4-be84-42ec-a55e-6e983e772380 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:37.325200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6dcd4-be84-42ec-a55e-6e983e772380 None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:37.325504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6dcd4-be84-42ec-a55e-6e983e772380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:37.326891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6dcd4-be84-42ec-a55e-6e983e772380 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:37.336758 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 145/298] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:37.345954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba70c1-79f9-4a6c-8f6c-2e17c015cab6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:37.413729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:37.460122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba70c1-79f9-4a6c-8f6c-2e17c015cab6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:37.494673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c424b-ba3b-4e97-8d1c-0366d5506bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:37.495597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c424b-ba3b-4e97-8d1c-0366d5506bd9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:37.495882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c424b-ba3b-4e97-8d1c-0366d5506bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:37.496160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c424b-ba3b-4e97-8d1c-0366d5506bd9 None admin] PATH_INFO: `/v3/users/e5013b666a824564b7b9b1009c09a44b` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:37.497409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c424b-ba3b-4e97-8d1c-0366d5506bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=e5013b666a824564b7b9b1009c09a44b)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:37.498875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c424b-ba3b-4e97-8d1c-0366d5506bd9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:37.500896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 154/299] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:37 2022] GET /identity/v3/users/e5013b666a824564b7b9b1009c09a44b => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:37.506642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6dcd4-be84-42ec-a55e-6e983e772380 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:37.545532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac886a2e-6962-4844-b5f5-2c00b6caa43c None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:37.546145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac886a2e-6962-4844-b5f5-2c00b6caa43c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:37.546630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac886a2e-6962-4844-b5f5-2c00b6caa43c None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:37.548775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 155/300] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:37.553813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240a72c-9515-4741-8fa3-5124d152124e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:37.554348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240a72c-9515-4741-8fa3-5124d152124e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:37.554828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240a72c-9515-4741-8fa3-5124d152124e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:37.571760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-1240a72c-9515-4741-8fa3-5124d152124e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:37.573300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1240a72c-9515-4741-8fa3-5124d152124e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:37.601313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:37.618579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6dcd4-be84-42ec-a55e-6e983e772380 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:37.649914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 156/301] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:37.660610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670b2f2-e3e0-4188-93bd-dc435ee1e0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:37.660610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670b2f2-e3e0-4188-93bd-dc435ee1e0a9 None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:37.660610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670b2f2-e3e0-4188-93bd-dc435ee1e0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:37.660610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670b2f2-e3e0-4188-93bd-dc435ee1e0a9 None admin] PATH_INFO: `/v3/system/users/e5013b666a824564b7b9b1009c09a44b/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:37.661743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670b2f2-e3e0-4188-93bd-dc435ee1e0a9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e5013b666a824564b7b9b1009c09a44b, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:37.663144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670b2f2-e3e0-4188-93bd-dc435ee1e0a9 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:37.675134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240a72c-9515-4741-8fa3-5124d152124e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:37.675805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 146/302] 10.222.0.47 () {58 vars in 1334 bytes} [Fri Jun 24 23:35:37 2022] PUT /identity/v3/system/users/e5013b666a824564b7b9b1009c09a44b/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:37.738049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:37.747045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a1d86-0934-4ecb-9bc8-294e8ad9885c None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:37.747674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a1d86-0934-4ecb-9bc8-294e8ad9885c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:37.748153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a1d86-0934-4ecb-9bc8-294e8ad9885c None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:37.750041 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 147/303] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:35:37.756454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016b2de-d6ee-48de-ba7c-ed9c8be61f1a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:37.756784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016b2de-d6ee-48de-ba7c-ed9c8be61f1a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:37.757069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016b2de-d6ee-48de-ba7c-ed9c8be61f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:37.779483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-5016b2de-d6ee-48de-ba7c-ed9c8be61f1a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:37.780836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5016b2de-d6ee-48de-ba7c-ed9c8be61f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:37.794796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240a72c-9515-4741-8fa3-5124d152124e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:37.839186 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e6d55-d178-4037-b632-dd784986748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:37.840108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e6d55-d178-4037-b632-dd784986748d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:37.840468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e6d55-d178-4037-b632-dd784986748d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:37.840840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e6d55-d178-4037-b632-dd784986748d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:37.854085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e6d55-d178-4037-b632-dd784986748d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-736e6d55-d178-4037-b632-dd784986748d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:37.863402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:37.864489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:37.865523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 157/304] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:37.870856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e6d55-d178-4037-b632-dd784986748d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:37.880182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 148/305] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:37.908641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016b2de-d6ee-48de-ba7c-ed9c8be61f1a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:38.003907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e6d55-d178-4037-b632-dd784986748d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:38.020427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:38.021791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:38.047171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45b64b-68d8-4259-a865-8d08e9891791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:38.049147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45b64b-68d8-4259-a865-8d08e9891791 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.049628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45b64b-68d8-4259-a865-8d08e9891791 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.050092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45b64b-68d8-4259-a865-8d08e9891791 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.050507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016b2de-d6ee-48de-ba7c-ed9c8be61f1a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:38.051278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45b64b-68d8-4259-a865-8d08e9891791 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:38.051787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45b64b-68d8-4259-a865-8d08e9891791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:38.053380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45b64b-68d8-4259-a865-8d08e9891791 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:38.062275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 158/306] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:38.068463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45b64b-68d8-4259-a865-8d08e9891791 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:38.089453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e303962-5fef-4f47-8bbb-6b2c8df7849f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:38.090263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e303962-5fef-4f47-8bbb-6b2c8df7849f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.090548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e303962-5fef-4f47-8bbb-6b2c8df7849f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.090825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e303962-5fef-4f47-8bbb-6b2c8df7849f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.092193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e303962-5fef-4f47-8bbb-6b2c8df7849f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:38.093503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e303962-5fef-4f47-8bbb-6b2c8df7849f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:38.100523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 149/307] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:38.101722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e303962-5fef-4f47-8bbb-6b2c8df7849f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:38.205492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45b64b-68d8-4259-a865-8d08e9891791 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:38.225712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e303962-5fef-4f47-8bbb-6b2c8df7849f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:38.242142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db019a9e-1663-4283-8fe3-a0b5a1205c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:38.242942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db019a9e-1663-4283-8fe3-a0b5a1205c46 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.243226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db019a9e-1663-4283-8fe3-a0b5a1205c46 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.243515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db019a9e-1663-4283-8fe3-a0b5a1205c46 None admin] PATH_INFO: `/v3/users/bf64d72b55344d809157e184b25b5b38` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.257976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db019a9e-1663-4283-8fe3-a0b5a1205c46 None admin] RBAC: Authorizing `identity:get_user(user_id=bf64d72b55344d809157e184b25b5b38)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:38.259752 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db019a9e-1663-4283-8fe3-a0b5a1205c46 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:38.266111 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 159/308] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:38 2022] GET /identity/v3/users/bf64d72b55344d809157e184b25b5b38 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:38.267504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db019a9e-1663-4283-8fe3-a0b5a1205c46 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:38.272681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f5d76-d932-4c4a-b7e4-cbf98bb9fe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:38.273862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f5d76-d932-4c4a-b7e4-cbf98bb9fe6c None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.274336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f5d76-d932-4c4a-b7e4-cbf98bb9fe6c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.274826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f5d76-d932-4c4a-b7e4-cbf98bb9fe6c None admin] PATH_INFO: `/v3/groups` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.275702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f5d76-d932-4c4a-b7e4-cbf98bb9fe6c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:38.277843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f5d76-d932-4c4a-b7e4-cbf98bb9fe6c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:38.304949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 150/309] 10.222.0.47 () {60 vars in 1134 bytes} [Fri Jun 24 23:35:38 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:35:38.312193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:38.343969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdb1d0-e790-4d8c-94fb-ad8c6cacc98f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.344321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdb1d0-e790-4d8c-94fb-ad8c6cacc98f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.344698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdb1d0-e790-4d8c-94fb-ad8c6cacc98f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.350187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 151/310] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:35:38.352094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698df49b-2fd3-4273-b2d8-1bef672ceb57 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.352384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698df49b-2fd3-4273-b2d8-1bef672ceb57 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.352694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698df49b-2fd3-4273-b2d8-1bef672ceb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.371558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-698df49b-2fd3-4273-b2d8-1bef672ceb57 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:38.372818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-698df49b-2fd3-4273-b2d8-1bef672ceb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:38.378312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:38.402978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db019a9e-1663-4283-8fe3-a0b5a1205c46 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:38.458971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a05bd1-231a-44bf-a3f7-1f03bc99883a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:38.459659 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a05bd1-231a-44bf-a3f7-1f03bc99883a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.459958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a05bd1-231a-44bf-a3f7-1f03bc99883a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.460481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a05bd1-231a-44bf-a3f7-1f03bc99883a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.461373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a05bd1-231a-44bf-a3f7-1f03bc99883a None admin] RBAC: Adding query filter params (role.id=3f15101d95de479cacc1f6d91a2ff504, user.id=bf64d72b55344d809157e184b25b5b38, scope.system=all) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:38.461687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a05bd1-231a-44bf-a3f7-1f03bc99883a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:38.463404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a05bd1-231a-44bf-a3f7-1f03bc99883a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:38.465173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 152/311] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:38.469450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 160/312] 10.222.0.47 () {56 vars in 1308 bytes} [Fri Jun 24 23:35:38 2022] GET /identity/v3/role_assignments?user.id=bf64d72b55344d809157e184b25b5b38&scope.system=all&role.id=3f15101d95de479cacc1f6d91a2ff504 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:38.489334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698df49b-2fd3-4273-b2d8-1bef672ceb57 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:38.603780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698df49b-2fd3-4273-b2d8-1bef672ceb57 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:38.642872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bac84a-0491-495b-8946-e84dade6525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:38.643670 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bac84a-0491-495b-8946-e84dade6525b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.643952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bac84a-0491-495b-8946-e84dade6525b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.644226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bac84a-0491-495b-8946-e84dade6525b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.651577 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:38.655337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bac84a-0491-495b-8946-e84dade6525b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-04bac84a-0491-495b-8946-e84dade6525b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:38.664125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:38.665104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:38.665863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:38.671313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 153/313] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:38.673513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a05bd1-231a-44bf-a3f7-1f03bc99883a None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:38.771754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:38.775220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a05bd1-231a-44bf-a3f7-1f03bc99883a None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:38.811445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b63365-31cd-48db-b07e-1db30df3a0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:38.812122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b63365-31cd-48db-b07e-1db30df3a0d5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.812402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b63365-31cd-48db-b07e-1db30df3a0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.812715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b63365-31cd-48db-b07e-1db30df3a0d5 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.814003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b63365-31cd-48db-b07e-1db30df3a0d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:38.814557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b63365-31cd-48db-b07e-1db30df3a0d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:38.815961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b63365-31cd-48db-b07e-1db30df3a0d5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:38.824864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 161/314] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:38.830292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bac84a-0491-495b-8946-e84dade6525b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:38.940658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bac84a-0491-495b-8946-e84dade6525b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:38.975455 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe35a1-9991-4eb9-acf9-ede773c87917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:38.976330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe35a1-9991-4eb9-acf9-ede773c87917 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:38.976630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe35a1-9991-4eb9-acf9-ede773c87917 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:38.976935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe35a1-9991-4eb9-acf9-ede773c87917 None admin] PATH_INFO: `/v3/users/c0b5fd91f511465282a9f388ee1eadef` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:38.978101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe35a1-9991-4eb9-acf9-ede773c87917 None admin] RBAC: Authorizing `identity:get_user(user_id=c0b5fd91f511465282a9f388ee1eadef)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:38.979567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe35a1-9991-4eb9-acf9-ede773c87917 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:38.981509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 154/315] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:38 2022] GET /identity/v3/users/c0b5fd91f511465282a9f388ee1eadef => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:38.987778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b63365-31cd-48db-b07e-1db30df3a0d5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:39.090537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b63365-31cd-48db-b07e-1db30df3a0d5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:39.124499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92670655-dc06-4b1a-bf85-7f92925ea258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:39.125186 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670655-dc06-4b1a-bf85-7f92925ea258 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:39.125474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670655-dc06-4b1a-bf85-7f92925ea258 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:39.125813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670655-dc06-4b1a-bf85-7f92925ea258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:39.157455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:39.196784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92670655-dc06-4b1a-bf85-7f92925ea258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:39.198259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92670655-dc06-4b1a-bf85-7f92925ea258 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:39.302577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 162/316] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:39.307889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe35a1-9991-4eb9-acf9-ede773c87917 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:39.373509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:39.417869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe35a1-9991-4eb9-acf9-ede773c87917 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:39.455963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81daf2d-160d-41e5-b24e-bda04b77ae1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:39.456872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81daf2d-160d-41e5-b24e-bda04b77ae1e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:39.457159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81daf2d-160d-41e5-b24e-bda04b77ae1e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:39.457441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81daf2d-160d-41e5-b24e-bda04b77ae1e None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:39.458644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81daf2d-160d-41e5-b24e-bda04b77ae1e None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:39.460107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81daf2d-160d-41e5-b24e-bda04b77ae1e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:39.462163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 155/317] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:39 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:39.467534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92670655-dc06-4b1a-bf85-7f92925ea258 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:39.529771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:39.569929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92670655-dc06-4b1a-bf85-7f92925ea258 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:39.611953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d5285-976b-4adb-9360-5438e8b50bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:39.612647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d5285-976b-4adb-9360-5438e8b50bba None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:39.612935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d5285-976b-4adb-9360-5438e8b50bba None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:39.613250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d5285-976b-4adb-9360-5438e8b50bba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:39.614125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d5285-976b-4adb-9360-5438e8b50bba None admin] RBAC: Adding query filter params (role.id=3f15101d95de479cacc1f6d91a2ff504, scope.project.id=9b25ac538fb448279a63db11f84f70de, user.id=c0b5fd91f511465282a9f388ee1eadef) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:39.614442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d5285-976b-4adb-9360-5438e8b50bba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:39.615923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d5285-976b-4adb-9360-5438e8b50bba None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:39.622628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 163/318] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:35:39 2022] GET /identity/v3/role_assignments?user.id=c0b5fd91f511465282a9f388ee1eadef&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=3f15101d95de479cacc1f6d91a2ff504 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:39.854944 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc352fe-01ec-4392-a611-006f8eef89f3 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:39.855604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc352fe-01ec-4392-a611-006f8eef89f3 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:39.856088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc352fe-01ec-4392-a611-006f8eef89f3 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:39.857710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 156/319] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:39.863654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c8c9d-2f9c-4477-bd52-8b0ad8375c3b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:39.864200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c8c9d-2f9c-4477-bd52-8b0ad8375c3b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:39.864715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c8c9d-2f9c-4477-bd52-8b0ad8375c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:39.881392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-db1c8c9d-2f9c-4477-bd52-8b0ad8375c3b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:39.882560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-db1c8c9d-2f9c-4477-bd52-8b0ad8375c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:39.948693 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 164/320] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:39.972726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc352fe-01ec-4392-a611-006f8eef89f3 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:40.081075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc352fe-01ec-4392-a611-006f8eef89f3 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:40.114863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9da86d-f836-4a4b-9db7-3f2ebfa47286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:40.115592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9da86d-f836-4a4b-9db7-3f2ebfa47286 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.115877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9da86d-f836-4a4b-9db7-3f2ebfa47286 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.116155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9da86d-f836-4a4b-9db7-3f2ebfa47286 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.118079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:40.129054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9da86d-f836-4a4b-9db7-3f2ebfa47286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-7a9da86d-f836-4a4b-9db7-3f2ebfa47286 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:40.136693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:40.137552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:40.138308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 157/321] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:40.144310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c8c9d-2f9c-4477-bd52-8b0ad8375c3b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:40.201837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:40.246990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c8c9d-2f9c-4477-bd52-8b0ad8375c3b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:40.280223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baca5f1-7ffb-4142-810b-47196d3076cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:40.280916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baca5f1-7ffb-4142-810b-47196d3076cd None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.281197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baca5f1-7ffb-4142-810b-47196d3076cd None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.281472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baca5f1-7ffb-4142-810b-47196d3076cd None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.282749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baca5f1-7ffb-4142-810b-47196d3076cd None admin] RBAC: Adding query filter params (name=member) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:40.283050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baca5f1-7ffb-4142-810b-47196d3076cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:40.284422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baca5f1-7ffb-4142-810b-47196d3076cd None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:40.292934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 165/322] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:40.299201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9da86d-f836-4a4b-9db7-3f2ebfa47286 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:40.403798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9da86d-f836-4a4b-9db7-3f2ebfa47286 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:40.436807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaca9f-31f4-4b3c-a4c0-94c131e97598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:40.437681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaca9f-31f4-4b3c-a4c0-94c131e97598 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.437963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaca9f-31f4-4b3c-a4c0-94c131e97598 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.438271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaca9f-31f4-4b3c-a4c0-94c131e97598 None admin] PATH_INFO: `/v3/users/e5013b666a824564b7b9b1009c09a44b` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.439488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aaca9f-31f4-4b3c-a4c0-94c131e97598 None admin] RBAC: Authorizing `identity:get_user(user_id=e5013b666a824564b7b9b1009c09a44b)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:40.440955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aaca9f-31f4-4b3c-a4c0-94c131e97598 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:40.442739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 158/323] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:40 2022] GET /identity/v3/users/e5013b666a824564b7b9b1009c09a44b => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:40.448557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baca5f1-7ffb-4142-810b-47196d3076cd None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:40.539768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c479cd-d775-4997-a834-53501d403238 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.540065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c479cd-d775-4997-a834-53501d403238 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.540347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c479cd-d775-4997-a834-53501d403238 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.541743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 159/324] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:35:40.546544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ea17b-09cd-4a16-862c-7acb46c5807e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.546835 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ea17b-09cd-4a16-862c-7acb46c5807e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.547117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ea17b-09cd-4a16-862c-7acb46c5807e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.551556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baca5f1-7ffb-4142-810b-47196d3076cd None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:40.563697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b06ea17b-09cd-4a16-862c-7acb46c5807e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:40.564940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b06ea17b-09cd-4a16-862c-7acb46c5807e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:40.570575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:40.589202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b4a6-f3be-461f-9048-73ec5efaa6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:40.589860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b4a6-f3be-461f-9048-73ec5efaa6f5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.590148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b4a6-f3be-461f-9048-73ec5efaa6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.590430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b4a6-f3be-461f-9048-73ec5efaa6f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.591256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961b4a6-f3be-461f-9048-73ec5efaa6f5 None admin] RBAC: Adding query filter params (role.id=86c49ec8587244e48fab81862ddb3641, user.id=e5013b666a824564b7b9b1009c09a44b, scope.system=all) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:40.591565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961b4a6-f3be-461f-9048-73ec5efaa6f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:40.593011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961b4a6-f3be-461f-9048-73ec5efaa6f5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:40.606397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 166/325] 10.222.0.47 () {56 vars in 1308 bytes} [Fri Jun 24 23:35:40 2022] GET /identity/v3/role_assignments?user.id=e5013b666a824564b7b9b1009c09a44b&scope.system=all&role.id=86c49ec8587244e48fab81862ddb3641 => generated 510 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:40.627265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:40.640394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 160/326] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:40.664255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b4a6-f3be-461f-9048-73ec5efaa6f5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:40.675989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96942b-f825-4d8a-8e9c-35e1369983d0 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.676329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96942b-f825-4d8a-8e9c-35e1369983d0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.676553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96942b-f825-4d8a-8e9c-35e1369983d0 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.678162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 161/327] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:40.683617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8d4a7-80ba-4be6-87de-584c9ebfbb40 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.683911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8d4a7-80ba-4be6-87de-584c9ebfbb40 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.684196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8d4a7-80ba-4be6-87de-584c9ebfbb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.701859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-90a8d4a7-80ba-4be6-87de-584c9ebfbb40 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:40.703085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-90a8d4a7-80ba-4be6-87de-584c9ebfbb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:40.777974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 162/328] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:40.784655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b4a6-f3be-461f-9048-73ec5efaa6f5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:40.800939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a8d4a7-80ba-4be6-87de-584c9ebfbb40 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:40.820976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd9097-de71-4da1-9563-763abe633891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:40.821772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd9097-de71-4da1-9563-763abe633891 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.822052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd9097-de71-4da1-9563-763abe633891 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.822323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd9097-de71-4da1-9563-763abe633891 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.823492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd9097-de71-4da1-9563-763abe633891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:40.824985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd9097-de71-4da1-9563-763abe633891 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:40.826596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 167/329] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:40.832318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd9097-de71-4da1-9563-763abe633891 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:40.878386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:40.902453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a8d4a7-80ba-4be6-87de-584c9ebfbb40 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:40.925371 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:40.932676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd9097-de71-4da1-9563-763abe633891 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:40.938881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4216ccb-31b8-43e1-9e8b-f8f9d3375a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:40.940119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4216ccb-31b8-43e1-9e8b-f8f9d3375a65 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.940413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4216ccb-31b8-43e1-9e8b-f8f9d3375a65 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.940720 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4216ccb-31b8-43e1-9e8b-f8f9d3375a65 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.950872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4216ccb-31b8-43e1-9e8b-f8f9d3375a65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-e4216ccb-31b8-43e1-9e8b-f8f9d3375a65 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:40.958806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:40.959683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:40.961009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 163/330] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:40.965646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4216ccb-31b8-43e1-9e8b-f8f9d3375a65 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:40.970465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df0d16-d60a-4a1c-a56d-ec0349949c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:40.971139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df0d16-d60a-4a1c-a56d-ec0349949c45 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:40.971422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df0d16-d60a-4a1c-a56d-ec0349949c45 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:40.971696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df0d16-d60a-4a1c-a56d-ec0349949c45 None admin] PATH_INFO: `/v3/groups` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:40.972386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df0d16-d60a-4a1c-a56d-ec0349949c45 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:40.973855 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df0d16-d60a-4a1c-a56d-ec0349949c45 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:40.989964 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 168/331] 10.222.0.47 () {60 vars in 1134 bytes} [Fri Jun 24 23:35:40 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:35:41.074081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4216ccb-31b8-43e1-9e8b-f8f9d3375a65 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:41.109468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6dd1b-4f3b-425f-8273-0e97171e36cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:41.110695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6dd1b-4f3b-425f-8273-0e97171e36cd None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:41.111054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6dd1b-4f3b-425f-8273-0e97171e36cd None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:41.111335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6dd1b-4f3b-425f-8273-0e97171e36cd None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:41.112096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6dd1b-4f3b-425f-8273-0e97171e36cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:41.112405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6dd1b-4f3b-425f-8273-0e97171e36cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:41.113836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6dd1b-4f3b-425f-8273-0e97171e36cd None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:41.122985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 164/332] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:41.128872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df0d16-d60a-4a1c-a56d-ec0349949c45 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:41.233035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df0d16-d60a-4a1c-a56d-ec0349949c45 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:41.235799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:41.272067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dae57b-73da-4b39-9c3f-483906ced57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:41.273242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dae57b-73da-4b39-9c3f-483906ced57e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:41.273519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dae57b-73da-4b39-9c3f-483906ced57e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:41.273839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dae57b-73da-4b39-9c3f-483906ced57e None admin] PATH_INFO: `/v3/users/bf64d72b55344d809157e184b25b5b38` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:41.275194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dae57b-73da-4b39-9c3f-483906ced57e None admin] RBAC: Authorizing `identity:get_user(user_id=bf64d72b55344d809157e184b25b5b38)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:41.276709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dae57b-73da-4b39-9c3f-483906ced57e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:41.278527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 169/333] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:41 2022] GET /identity/v3/users/bf64d72b55344d809157e184b25b5b38 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:41.283990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6dd1b-4f3b-425f-8273-0e97171e36cd None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:41.334673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:41.394422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6dd1b-4f3b-425f-8273-0e97171e36cd None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:41.429659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68c1a0-67e8-4e3e-b81f-60255e97e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:41.430535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68c1a0-67e8-4e3e-b81f-60255e97e51a None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:41.430817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68c1a0-67e8-4e3e-b81f-60255e97e51a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:41.431088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68c1a0-67e8-4e3e-b81f-60255e97e51a None admin] PATH_INFO: `/v3/system/users/bf64d72b55344d809157e184b25b5b38/roles/3f15101d95de479cacc1f6d91a2ff504` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:41.432635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68c1a0-67e8-4e3e-b81f-60255e97e51a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bf64d72b55344d809157e184b25b5b38, role_id=3f15101d95de479cacc1f6d91a2ff504)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:41.434005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68c1a0-67e8-4e3e-b81f-60255e97e51a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:41.443456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 165/334] 10.222.0.47 () {58 vars in 1334 bytes} [Fri Jun 24 23:35:41 2022] PUT /identity/v3/system/users/bf64d72b55344d809157e184b25b5b38/roles/3f15101d95de479cacc1f6d91a2ff504 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:43.839448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ff35d-c88b-41ed-92f2-4f51eac4f16c None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:43.839960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ff35d-c88b-41ed-92f2-4f51eac4f16c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:43.840034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ff35d-c88b-41ed-92f2-4f51eac4f16c None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:43.841620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 170/335] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:43.857340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcca480-44a0-4f99-b90e-92c90d962f80 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:43.858269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcca480-44a0-4f99-b90e-92c90d962f80 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:43.858750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcca480-44a0-4f99-b90e-92c90d962f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:43.881354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7dcca480-44a0-4f99-b90e-92c90d962f80 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:43.883020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7dcca480-44a0-4f99-b90e-92c90d962f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:43.978476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 166/336] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:44.021459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ff35d-c88b-41ed-92f2-4f51eac4f16c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:44.134790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19fb6c-4279-4694-aec0-1bd99e6c6242 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.135342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19fb6c-4279-4694-aec0-1bd99e6c6242 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.135768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19fb6c-4279-4694-aec0-1bd99e6c6242 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.137597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 167/337] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:35:44.155217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd452fc-5ba9-41e6-9f27-a8b6eae74c1c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.155724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd452fc-5ba9-41e6-9f27-a8b6eae74c1c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.156174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd452fc-5ba9-41e6-9f27-a8b6eae74c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.172001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ff35d-c88b-41ed-92f2-4f51eac4f16c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:44.186857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0fd452fc-5ba9-41e6-9f27-a8b6eae74c1c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:44.188462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0fd452fc-5ba9-41e6-9f27-a8b6eae74c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:44.222694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaab22e-b068-4eac-9e42-87edc963febc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:44.223499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaab22e-b068-4eac-9e42-87edc963febc None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.223782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaab22e-b068-4eac-9e42-87edc963febc None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.224058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaab22e-b068-4eac-9e42-87edc963febc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.240940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:44.250366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaab22e-b068-4eac-9e42-87edc963febc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ffaab22e-b068-4eac-9e42-87edc963febc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:44.258610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:44.261319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:44.262319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:44.272632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 171/338] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:44.273815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaab22e-b068-4eac-9e42-87edc963febc None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:44.316861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 168/339] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:44.336944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:44.378045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd452fc-5ba9-41e6-9f27-a8b6eae74c1c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:44.428915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaab22e-b068-4eac-9e42-87edc963febc None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:44.473251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616f19d-f48d-430b-a92e-89c0c560dcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:44.473940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f19d-f48d-430b-a92e-89c0c560dcba None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.474225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f19d-f48d-430b-a92e-89c0c560dcba None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.474507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616f19d-f48d-430b-a92e-89c0c560dcba None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.475279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616f19d-f48d-430b-a92e-89c0c560dcba None admin] RBAC: Adding query filter params (name=member) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:44.475590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616f19d-f48d-430b-a92e-89c0c560dcba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:44.477035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616f19d-f48d-430b-a92e-89c0c560dcba None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:44.486864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 172/340] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:44.488233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58169dfe-2b3e-40bc-bbf2-d5efb5da1004 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.488530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58169dfe-2b3e-40bc-bbf2-d5efb5da1004 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.488848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58169dfe-2b3e-40bc-bbf2-d5efb5da1004 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.490368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 173/341] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:44.491450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cea96-1af4-4c60-a075-4fa4cbf5a54f None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.491759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cea96-1af4-4c60-a075-4fa4cbf5a54f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.492019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cea96-1af4-4c60-a075-4fa4cbf5a54f None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.493386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 174/342] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:44.499256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623be2b3-2271-4bcd-80bd-ccfaa500d90e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.499547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623be2b3-2271-4bcd-80bd-ccfaa500d90e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.500259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623be2b3-2271-4bcd-80bd-ccfaa500d90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.527946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-623be2b3-2271-4bcd-80bd-ccfaa500d90e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:44.529517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-623be2b3-2271-4bcd-80bd-ccfaa500d90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:44.554403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd452fc-5ba9-41e6-9f27-a8b6eae74c1c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:44.598727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a49a2-b215-4e2a-8a90-6d6abcc33866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:44.599981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a49a2-b215-4e2a-8a90-6d6abcc33866 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.600485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a49a2-b215-4e2a-8a90-6d6abcc33866 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.600849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a49a2-b215-4e2a-8a90-6d6abcc33866 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.612537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a49a2-b215-4e2a-8a90-6d6abcc33866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-b52a49a2-b215-4e2a-8a90-6d6abcc33866 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:44.622342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:44.623498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:44.624500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:44.628350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 169/343] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:44.629294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a49a2-b215-4e2a-8a90-6d6abcc33866 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:44.632714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 175/344] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:44.636330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd11f6-1119-4012-b3db-44e03b5aae8d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.636330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd11f6-1119-4012-b3db-44e03b5aae8d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.636537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd11f6-1119-4012-b3db-44e03b5aae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.668417 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-abfd11f6-1119-4012-b3db-44e03b5aae8d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:44.669707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-abfd11f6-1119-4012-b3db-44e03b5aae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:44.679145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:44.695047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:44.757396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 176/345] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:44.758524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd11f6-1119-4012-b3db-44e03b5aae8d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:44.795446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a49a2-b215-4e2a-8a90-6d6abcc33866 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:44.853528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb8a54-92a4-4900-93f1-bc32f275ea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:44.853528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb8a54-92a4-4900-93f1-bc32f275ea96 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.853528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb8a54-92a4-4900-93f1-bc32f275ea96 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.853528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb8a54-92a4-4900-93f1-bc32f275ea96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.887678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd11f6-1119-4012-b3db-44e03b5aae8d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:44.936772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824665cf-759e-49a6-9c95-6389fbb2a831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:44.937382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824665cf-759e-49a6-9c95-6389fbb2a831 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:44.937663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824665cf-759e-49a6-9c95-6389fbb2a831 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:44.937938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824665cf-759e-49a6-9c95-6389fbb2a831 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:44.938855 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824665cf-759e-49a6-9c95-6389fbb2a831 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:44.939151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb8a54-92a4-4900-93f1-bc32f275ea96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:44.939559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824665cf-759e-49a6-9c95-6389fbb2a831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:44.940527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824665cf-759e-49a6-9c95-6389fbb2a831 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:44.940718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb8a54-92a4-4900-93f1-bc32f275ea96 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:44.950694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 177/346] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:44.951754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824665cf-759e-49a6-9c95-6389fbb2a831 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:44.976387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:44.981906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:45.079344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 170/347] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:45.080545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2bc78-b270-4990-b9d9-c3d05345cf05 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.081813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2bc78-b270-4990-b9d9-c3d05345cf05 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.082268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2bc78-b270-4990-b9d9-c3d05345cf05 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.083755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 171/348] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:45.083881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824665cf-759e-49a6-9c95-6389fbb2a831 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:45.084825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2bc78-b270-4990-b9d9-c3d05345cf05 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:45.126547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2ce4a-19eb-4b19-ba11-57d39c8a24ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:45.127367 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2ce4a-19eb-4b19-ba11-57d39c8a24ce None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.127661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2ce4a-19eb-4b19-ba11-57d39c8a24ce None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.127941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2ce4a-19eb-4b19-ba11-57d39c8a24ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.139970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2ce4a-19eb-4b19-ba11-57d39c8a24ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-28a2ce4a-19eb-4b19-ba11-57d39c8a24ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:45.149676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:45.150594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 23:35:45.151477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:45.153617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 178/349] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:45.155579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2ce4a-19eb-4b19-ba11-57d39c8a24ce None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:45.223770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2bc78-b270-4990-b9d9-c3d05345cf05 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:45.247079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:45.255186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:45.264423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654f0e8-036b-4c7f-8a5d-49b6b4b1330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:45.265338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654f0e8-036b-4c7f-8a5d-49b6b4b1330d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.265616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654f0e8-036b-4c7f-8a5d-49b6b4b1330d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.265899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654f0e8-036b-4c7f-8a5d-49b6b4b1330d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.280016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2ce4a-19eb-4b19-ba11-57d39c8a24ce None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:45.284246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654f0e8-036b-4c7f-8a5d-49b6b4b1330d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-3654f0e8-036b-4c7f-8a5d-49b6b4b1330d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:45.294199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:45.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:45.296610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 172/350] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:45.302036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed3b32-e57d-4850-80f6-464d732253ba None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.302327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed3b32-e57d-4850-80f6-464d732253ba None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.302604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed3b32-e57d-4850-80f6-464d732253ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.322414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-17ed3b32-e57d-4850-80f6-464d732253ba None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:45.323829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-17ed3b32-e57d-4850-80f6-464d732253ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:45.325364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e27a-5057-4872-8fc8-6e0d6d8207b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:45.326038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e27a-5057-4872-8fc8-6e0d6d8207b2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.327050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e27a-5057-4872-8fc8-6e0d6d8207b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.327346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e27a-5057-4872-8fc8-6e0d6d8207b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.413307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564e27a-5057-4872-8fc8-6e0d6d8207b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:45.413523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 173/351] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:45.414210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed3b32-e57d-4850-80f6-464d732253ba None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:45.415276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564e27a-5057-4872-8fc8-6e0d6d8207b2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:45.505582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:45.539584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 179/352] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:45.540576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e27a-5057-4872-8fc8-6e0d6d8207b2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:45.552484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed3b32-e57d-4850-80f6-464d732253ba None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:45.571626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:45.591700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350f2e1-d5a4-4392-913d-0ec6514d8a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:45.592841 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350f2e1-d5a4-4392-913d-0ec6514d8a8b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.593268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350f2e1-d5a4-4392-913d-0ec6514d8a8b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.593719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350f2e1-d5a4-4392-913d-0ec6514d8a8b None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.596279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350f2e1-d5a4-4392-913d-0ec6514d8a8b None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:45.597866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350f2e1-d5a4-4392-913d-0ec6514d8a8b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:45.599601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 174/353] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:45 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:45.600875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a86ed3-ffe5-479d-b674-7da4b2d7561d None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.601123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a86ed3-ffe5-479d-b674-7da4b2d7561d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.601391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a86ed3-ffe5-479d-b674-7da4b2d7561d None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.602801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 175/354] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:45.603696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a86ed3-ffe5-479d-b674-7da4b2d7561d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:45.653139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e27a-5057-4872-8fc8-6e0d6d8207b2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:45.695615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362337b-887a-4818-ba3f-4ad1f2ca47eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:45.696358 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362337b-887a-4818-ba3f-4ad1f2ca47eb None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.697109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362337b-887a-4818-ba3f-4ad1f2ca47eb None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.697109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362337b-887a-4818-ba3f-4ad1f2ca47eb None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.697714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362337b-887a-4818-ba3f-4ad1f2ca47eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:45.698028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362337b-887a-4818-ba3f-4ad1f2ca47eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:45.701246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362337b-887a-4818-ba3f-4ad1f2ca47eb None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:45.708445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 180/355] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:45.709313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362337b-887a-4818-ba3f-4ad1f2ca47eb None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:45.727455 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a86ed3-ffe5-479d-b674-7da4b2d7561d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:45.755415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:45.775748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b27186-2148-409c-a59e-7235055df692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:45.776670 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b27186-2148-409c-a59e-7235055df692 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.776957 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b27186-2148-409c-a59e-7235055df692 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.777234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b27186-2148-409c-a59e-7235055df692 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.778157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b27186-2148-409c-a59e-7235055df692 None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:45.778931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b27186-2148-409c-a59e-7235055df692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:45.780967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b27186-2148-409c-a59e-7235055df692 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:45.791615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 176/356] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:45.792544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b27186-2148-409c-a59e-7235055df692 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:45.851975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362337b-887a-4818-ba3f-4ad1f2ca47eb None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:45.857786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:45.898604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c47fe7-7d9a-46d9-8f06-64341c29d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:45.899986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c47fe7-7d9a-46d9-8f06-64341c29d019 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.900417 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c47fe7-7d9a-46d9-8f06-64341c29d019 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.900865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c47fe7-7d9a-46d9-8f06-64341c29d019 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.917285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c47fe7-7d9a-46d9-8f06-64341c29d019 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:45.920441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b27186-2148-409c-a59e-7235055df692 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-74c47fe7-7d9a-46d9-8f06-64341c29d019 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:45.926019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:45.927085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:45.927933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:45.929846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 181/357] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:45.930794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c47fe7-7d9a-46d9-8f06-64341c29d019 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:45.972425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692f991-f207-4ecc-93be-3e1af25cb551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:45.973383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692f991-f207-4ecc-93be-3e1af25cb551 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.973665 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692f991-f207-4ecc-93be-3e1af25cb551 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.973940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692f991-f207-4ecc-93be-3e1af25cb551 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:45.976624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692f991-f207-4ecc-93be-3e1af25cb551 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:45.977465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692f991-f207-4ecc-93be-3e1af25cb551 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:45.979162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 177/358] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:45 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:45.987683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a63865-1eaa-4af3-9687-60e82964130c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:45.987683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a63865-1eaa-4af3-9687-60e82964130c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:45.987683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a63865-1eaa-4af3-9687-60e82964130c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.004042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-92a63865-1eaa-4af3-9687-60e82964130c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:46.005265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-92a63865-1eaa-4af3-9687-60e82964130c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:46.056318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:46.060087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c47fe7-7d9a-46d9-8f06-64341c29d019 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.088201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 178/359] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:46.097444 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a63865-1eaa-4af3-9687-60e82964130c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:46.104352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc603a4-d507-4413-9cb1-93b06616ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.105265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc603a4-d507-4413-9cb1-93b06616ef50 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.105554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc603a4-d507-4413-9cb1-93b06616ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.105834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc603a4-d507-4413-9cb1-93b06616ef50 None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.115031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc603a4-d507-4413-9cb1-93b06616ef50 None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:46.117078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc603a4-d507-4413-9cb1-93b06616ef50 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:46.119085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 182/360] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:45 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:46.119751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc603a4-d507-4413-9cb1-93b06616ef50 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:46.161589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:46.227495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a63865-1eaa-4af3-9687-60e82964130c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.251803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc603a4-d507-4413-9cb1-93b06616ef50 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.271172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b775b43-eb7f-4352-9c13-4c0cbed47e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.272053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b775b43-eb7f-4352-9c13-4c0cbed47e7e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.272332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b775b43-eb7f-4352-9c13-4c0cbed47e7e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.272706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b775b43-eb7f-4352-9c13-4c0cbed47e7e None admin] PATH_INFO: `/v3/users/bf64d72b55344d809157e184b25b5b38` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.274161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b775b43-eb7f-4352-9c13-4c0cbed47e7e None admin] RBAC: Authorizing `identity:get_user(user_id=bf64d72b55344d809157e184b25b5b38)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:46.275606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b775b43-eb7f-4352-9c13-4c0cbed47e7e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:46.277263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 179/361] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/users/bf64d72b55344d809157e184b25b5b38 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:46.278016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b775b43-eb7f-4352-9c13-4c0cbed47e7e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:46.286471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbb492-fd83-4fc5-8c55-0a4f6d0ceaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.287125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbb492-fd83-4fc5-8c55-0a4f6d0ceaa5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.287408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbb492-fd83-4fc5-8c55-0a4f6d0ceaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.287686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbb492-fd83-4fc5-8c55-0a4f6d0ceaa5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.317058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:46.354603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbb492-fd83-4fc5-8c55-0a4f6d0ceaa5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:46.356074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbb492-fd83-4fc5-8c55-0a4f6d0ceaa5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:46.376258 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b775b43-eb7f-4352-9c13-4c0cbed47e7e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.390040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:46.410861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af2c1f-bacd-4093-84ce-45641bda4d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.411530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af2c1f-bacd-4093-84ce-45641bda4d3c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.411816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af2c1f-bacd-4093-84ce-45641bda4d3c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.412097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af2c1f-bacd-4093-84ce-45641bda4d3c None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.412877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af2c1f-bacd-4093-84ce-45641bda4d3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:46.413189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af2c1f-bacd-4093-84ce-45641bda4d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:46.414560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af2c1f-bacd-4093-84ce-45641bda4d3c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:46.422865 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 180/362] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:46.425439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af2c1f-bacd-4093-84ce-45641bda4d3c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:46.452040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 183/363] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:46.452900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbb492-fd83-4fc5-8c55-0a4f6d0ceaa5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:46.525333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:46.540553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af2c1f-bacd-4093-84ce-45641bda4d3c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.563898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbb492-fd83-4fc5-8c55-0a4f6d0ceaa5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.573164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00bff1-0cda-46a6-a2a7-e82e6b946beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.573966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00bff1-0cda-46a6-a2a7-e82e6b946beb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.574252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00bff1-0cda-46a6-a2a7-e82e6b946beb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.574531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00bff1-0cda-46a6-a2a7-e82e6b946beb None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.575711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00bff1-0cda-46a6-a2a7-e82e6b946beb None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:46.577210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e00bff1-0cda-46a6-a2a7-e82e6b946beb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:46.579346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 181/364] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:46.580109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00bff1-0cda-46a6-a2a7-e82e6b946beb None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:46.598808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1705-bb21-41af-8152-f745755fb946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.599609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1705-bb21-41af-8152-f745755fb946 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.599892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1705-bb21-41af-8152-f745755fb946 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.600170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1705-bb21-41af-8152-f745755fb946 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.610069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1705-bb21-41af-8152-f745755fb946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-8f8a1705-bb21-41af-8152-f745755fb946 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:46.617676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:46.618534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:46.619301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 184/365] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:46.619940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1705-bb21-41af-8152-f745755fb946 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:46.626912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:46.684798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00bff1-0cda-46a6-a2a7-e82e6b946beb None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.727606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1705-bb21-41af-8152-f745755fb946 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.730620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce32ae-0b4e-4576-97e6-e558998bf799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.731043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce32ae-0b4e-4576-97e6-e558998bf799 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.731327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce32ae-0b4e-4576-97e6-e558998bf799 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.731643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce32ae-0b4e-4576-97e6-e558998bf799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.732492 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce32ae-0b4e-4576-97e6-e558998bf799 None admin] RBAC: Adding query filter params (group.id=7c061376f4434aab8c840883a46f2fe1, role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:46.732840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce32ae-0b4e-4576-97e6-e558998bf799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:46.734285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce32ae-0b4e-4576-97e6-e558998bf799 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:46.740094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 182/366] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/role_assignments?group.id=7c061376f4434aab8c840883a46f2fe1&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=86c49ec8587244e48fab81862ddb3641 => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:46.745999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce32ae-0b4e-4576-97e6-e558998bf799 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:46.767565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:46.781080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707e8c3-ea90-406d-9f29-f302489b2806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.782744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707e8c3-ea90-406d-9f29-f302489b2806 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.783403 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707e8c3-ea90-406d-9f29-f302489b2806 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.783996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707e8c3-ea90-406d-9f29-f302489b2806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.784802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707e8c3-ea90-406d-9f29-f302489b2806 None admin] RBAC: Adding query filter params (role.id=3f15101d95de479cacc1f6d91a2ff504, user.id=bf64d72b55344d809157e184b25b5b38, scope.system=all) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:46.785493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707e8c3-ea90-406d-9f29-f302489b2806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:46.786780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707e8c3-ea90-406d-9f29-f302489b2806 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:46.810984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 185/367] 10.222.0.47 () {56 vars in 1308 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/role_assignments?user.id=bf64d72b55344d809157e184b25b5b38&scope.system=all&role.id=3f15101d95de479cacc1f6d91a2ff504 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:46.811716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707e8c3-ea90-406d-9f29-f302489b2806 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:46.878097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce32ae-0b4e-4576-97e6-e558998bf799 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.893146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:46.930599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707e8c3-ea90-406d-9f29-f302489b2806 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:46.935317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e54533-51b6-409d-b44b-0822ede5bb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.936015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e54533-51b6-409d-b44b-0822ede5bb63 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.936304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e54533-51b6-409d-b44b-0822ede5bb63 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.936610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e54533-51b6-409d-b44b-0822ede5bb63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.987570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77043bbb-0f87-4ac7-be8a-3fea4dec5d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:46.988508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77043bbb-0f87-4ac7-be8a-3fea4dec5d32 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:46.988890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77043bbb-0f87-4ac7-be8a-3fea4dec5d32 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:46.989109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77043bbb-0f87-4ac7-be8a-3fea4dec5d32 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:46.990430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77043bbb-0f87-4ac7-be8a-3fea4dec5d32 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:46.991895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77043bbb-0f87-4ac7-be8a-3fea4dec5d32 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:46.999187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 186/368] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:47.000191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77043bbb-0f87-4ac7-be8a-3fea4dec5d32 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:47.077658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e54533-51b6-409d-b44b-0822ede5bb63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:47.078176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e54533-51b6-409d-b44b-0822ede5bb63 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:47.085869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:47.175940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77043bbb-0f87-4ac7-be8a-3fea4dec5d32 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:47.224250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c494a8-cd82-4a23-98ed-457f71756147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:47.225549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c494a8-cd82-4a23-98ed-457f71756147 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:47.225837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c494a8-cd82-4a23-98ed-457f71756147 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:47.226273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c494a8-cd82-4a23-98ed-457f71756147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:47.227142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c494a8-cd82-4a23-98ed-457f71756147 None admin] RBAC: Adding query filter params (group.id=7c061376f4434aab8c840883a46f2fe1, role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:47.227440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c494a8-cd82-4a23-98ed-457f71756147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:47.229478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c494a8-cd82-4a23-98ed-457f71756147 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:47.231410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 183/369] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:47.232325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e54533-51b6-409d-b44b-0822ede5bb63 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:47.236346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 187/370] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:46 2022] GET /identity/v3/role_assignments?group.id=7c061376f4434aab8c840883a46f2fe1&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:47.237249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c494a8-cd82-4a23-98ed-457f71756147 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:47.250555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:47.374550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c494a8-cd82-4a23-98ed-457f71756147 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:47.380752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e54533-51b6-409d-b44b-0822ede5bb63 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:47.396876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:47.435616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29c7ed-2de9-4585-b919-fadfe855eaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:47.436413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29c7ed-2de9-4585-b919-fadfe855eaab None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:47.436732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29c7ed-2de9-4585-b919-fadfe855eaab None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:47.436955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143faee-8d5b-4e8d-ad76-c2a6a004188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:47.437087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29c7ed-2de9-4585-b919-fadfe855eaab None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:47.438156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143faee-8d5b-4e8d-ad76-c2a6a004188e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:47.438574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29c7ed-2de9-4585-b919-fadfe855eaab None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:47.439465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143faee-8d5b-4e8d-ad76-c2a6a004188e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:47.439936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143faee-8d5b-4e8d-ad76-c2a6a004188e None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:47.440051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d29c7ed-2de9-4585-b919-fadfe855eaab None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:47.441050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143faee-8d5b-4e8d-ad76-c2a6a004188e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:47.441561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143faee-8d5b-4e8d-ad76-c2a6a004188e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:47.441980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 188/371] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:47 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:47.442548 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29c7ed-2de9-4585-b919-fadfe855eaab None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:47.443810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143faee-8d5b-4e8d-ad76-c2a6a004188e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:47.457337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 184/372] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:47.458523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143faee-8d5b-4e8d-ad76-c2a6a004188e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:47.554302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:47.587415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29c7ed-2de9-4585-b919-fadfe855eaab None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:47.592128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143faee-8d5b-4e8d-ad76-c2a6a004188e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:47.630589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6568d08-42e5-4368-9d66-888a4923a5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:47.631256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6568d08-42e5-4368-9d66-888a4923a5db None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:47.631541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6568d08-42e5-4368-9d66-888a4923a5db None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:47.631822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6568d08-42e5-4368-9d66-888a4923a5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:47.632307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f025f-2049-4852-8b03-7e2cd45ca5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:47.632683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6568d08-42e5-4368-9d66-888a4923a5db None admin] RBAC: Adding query filter params (group.id=7c061376f4434aab8c840883a46f2fe1, role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=9b25ac538fb448279a63db11f84f70de) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:47.633003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6568d08-42e5-4368-9d66-888a4923a5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:47.633204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f025f-2049-4852-8b03-7e2cd45ca5e3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:47.633491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f025f-2049-4852-8b03-7e2cd45ca5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:47.633770 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f025f-2049-4852-8b03-7e2cd45ca5e3 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:47.634465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6568d08-42e5-4368-9d66-888a4923a5db None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:47.634957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f025f-2049-4852-8b03-7e2cd45ca5e3 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:47.636679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f025f-2049-4852-8b03-7e2cd45ca5e3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:47.639428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 189/373] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:47 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:47.640038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f025f-2049-4852-8b03-7e2cd45ca5e3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:47.640529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 185/374] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:47 2022] GET /identity/v3/role_assignments?group.id=7c061376f4434aab8c840883a46f2fe1&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=86c49ec8587244e48fab81862ddb3641 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:47.645640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6568d08-42e5-4368-9d66-888a4923a5db None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:47.744695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:47.836566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f025f-2049-4852-8b03-7e2cd45ca5e3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:47.858784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6568d08-42e5-4368-9d66-888a4923a5db None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:47.880092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a64877-b345-4d14-ac3a-fe4b87175868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:47.880799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a64877-b345-4d14-ac3a-fe4b87175868 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:47.881362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a64877-b345-4d14-ac3a-fe4b87175868 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:47.881647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a64877-b345-4d14-ac3a-fe4b87175868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:47.915375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8092-03b3-4c8d-9a22-87adb8e0f125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:47.916329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8092-03b3-4c8d-9a22-87adb8e0f125 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:47.916665 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8092-03b3-4c8d-9a22-87adb8e0f125 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:47.916955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8092-03b3-4c8d-9a22-87adb8e0f125 None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:47.918378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8092-03b3-4c8d-9a22-87adb8e0f125 None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:47.919836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8092-03b3-4c8d-9a22-87adb8e0f125 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:47.921974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 186/375] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:47 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:47.933703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8092-03b3-4c8d-9a22-87adb8e0f125 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:47.937355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:47.965090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a64877-b345-4d14-ac3a-fe4b87175868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:47.966552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a64877-b345-4d14-ac3a-fe4b87175868 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:48.057043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:48.077220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 190/376] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:48.083749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a64877-b345-4d14-ac3a-fe4b87175868 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:48.195902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a64877-b345-4d14-ac3a-fe4b87175868 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:48.232055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f672c-7c73-4d04-934e-646f400323f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:48.232955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f672c-7c73-4d04-934e-646f400323f6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:48.233238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f672c-7c73-4d04-934e-646f400323f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:48.233515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f672c-7c73-4d04-934e-646f400323f6 None admin] PATH_INFO: `/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:48.235520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f672c-7c73-4d04-934e-646f400323f6 None admin] RBAC: Authorizing `identity:get_project(project_id=1b6f1fb3ca4d42909a61ac0a3da08290)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:48.237605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f672c-7c73-4d04-934e-646f400323f6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:48.239898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 191/377] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:48 2022] GET /identity/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:48.240447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8092-03b3-4c8d-9a22-87adb8e0f125 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:48.246183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f672c-7c73-4d04-934e-646f400323f6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:48.276958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:48.319884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f41d9-fcdf-41fe-b82f-d470da5012c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:48.320947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f41d9-fcdf-41fe-b82f-d470da5012c5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:48.321443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f41d9-fcdf-41fe-b82f-d470da5012c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:48.321918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f41d9-fcdf-41fe-b82f-d470da5012c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:48.322989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f41d9-fcdf-41fe-b82f-d470da5012c5 None admin] RBAC: Adding query filter params (group.id=7c061376f4434aab8c840883a46f2fe1, role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=9b25ac538fb448279a63db11f84f70de) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:48.323502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f41d9-fcdf-41fe-b82f-d470da5012c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:48.325171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f41d9-fcdf-41fe-b82f-d470da5012c5 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:48.339323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 187/378] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:47 2022] GET /identity/v3/role_assignments?group.id=7c061376f4434aab8c840883a46f2fe1&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 260 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:48.364652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f672c-7c73-4d04-934e-646f400323f6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:48.401809 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9180c53-690b-4ea7-9018-3b09b572e27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:48.402672 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9180c53-690b-4ea7-9018-3b09b572e27d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:48.402952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9180c53-690b-4ea7-9018-3b09b572e27d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:48.403225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9180c53-690b-4ea7-9018-3b09b572e27d None admin] PATH_INFO: `/v3/groups/3f872f13c64448b389b1d9f4f8f34c41` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:48.413343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9180c53-690b-4ea7-9018-3b09b572e27d None admin] RBAC: Authorizing `identity:get_group(group_id=3f872f13c64448b389b1d9f4f8f34c41)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:48.414851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9180c53-690b-4ea7-9018-3b09b572e27d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:48.420907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 192/379] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:48 2022] GET /identity/v3/groups/3f872f13c64448b389b1d9f4f8f34c41 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:48.422969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f41d9-fcdf-41fe-b82f-d470da5012c5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:48.539611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:48.572276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f41d9-fcdf-41fe-b82f-d470da5012c5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:48.608457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5be8c-4894-49c2-85fa-0038415aeaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:48.609152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5be8c-4894-49c2-85fa-0038415aeaec None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:48.609441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5be8c-4894-49c2-85fa-0038415aeaec None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:48.609723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5be8c-4894-49c2-85fa-0038415aeaec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:48.610629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5be8c-4894-49c2-85fa-0038415aeaec None admin] RBAC: Adding query filter params (group.id=3f872f13c64448b389b1d9f4f8f34c41, role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=1b6f1fb3ca4d42909a61ac0a3da08290) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:48.610946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5be8c-4894-49c2-85fa-0038415aeaec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:48.612416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5be8c-4894-49c2-85fa-0038415aeaec None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:48.618487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 188/380] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:48 2022] GET /identity/v3/role_assignments?group.id=3f872f13c64448b389b1d9f4f8f34c41&scope.project.id=1b6f1fb3ca4d42909a61ac0a3da08290&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:49.054065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9ea02-c093-4512-b7bb-d6a24dd05905 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:49.055008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9ea02-c093-4512-b7bb-d6a24dd05905 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:49.055485 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9ea02-c093-4512-b7bb-d6a24dd05905 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:49.057352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 193/381] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:49.062502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45ec6c-b31d-470d-bf7f-e9fa720f3608 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:49.062798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45ec6c-b31d-470d-bf7f-e9fa720f3608 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:49.063739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45ec6c-b31d-470d-bf7f-e9fa720f3608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:49.089626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-cc45ec6c-b31d-470d-bf7f-e9fa720f3608 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:49.090854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-cc45ec6c-b31d-470d-bf7f-e9fa720f3608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:49.175517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 189/382] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:49.218122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9ea02-c093-4512-b7bb-d6a24dd05905 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:49.324238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:49.346695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9ea02-c093-4512-b7bb-d6a24dd05905 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:49.383129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afda768-ec04-4ff9-a8e4-b4ac4e8691e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:49.383934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afda768-ec04-4ff9-a8e4-b4ac4e8691e3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:49.384220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afda768-ec04-4ff9-a8e4-b4ac4e8691e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:49.384500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afda768-ec04-4ff9-a8e4-b4ac4e8691e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:49.395913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afda768-ec04-4ff9-a8e4-b4ac4e8691e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-9afda768-ec04-4ff9-a8e4-b4ac4e8691e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:49.403893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:49.404846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:49.405608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 194/383] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:49.411346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45ec6c-b31d-470d-bf7f-e9fa720f3608 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:49.535910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:49.575573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45ec6c-b31d-470d-bf7f-e9fa720f3608 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:49.628338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44b3f6-ecf7-4838-b7bc-a3893100e35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:49.629049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b3f6-ecf7-4838-b7bc-a3893100e35a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:49.629337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b3f6-ecf7-4838-b7bc-a3893100e35a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:49.629620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b3f6-ecf7-4838-b7bc-a3893100e35a None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:49.630385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44b3f6-ecf7-4838-b7bc-a3893100e35a None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:49.630699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44b3f6-ecf7-4838-b7bc-a3893100e35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:49.632077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44b3f6-ecf7-4838-b7bc-a3893100e35a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:49.641868 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 190/384] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:49.647544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afda768-ec04-4ff9-a8e4-b4ac4e8691e3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:49.712397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce3abf-2477-495d-85f9-c0c9dfdc6380 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:49.712607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce3abf-2477-495d-85f9-c0c9dfdc6380 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:49.712891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce3abf-2477-495d-85f9-c0c9dfdc6380 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:49.714632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 191/385] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:49.719343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c02cb-b596-421f-838c-0b4840da5571 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:49.719644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c02cb-b596-421f-838c-0b4840da5571 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:49.719920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c02cb-b596-421f-838c-0b4840da5571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:49.756677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-917c02cb-b596-421f-838c-0b4840da5571 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:49.759724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-917c02cb-b596-421f-838c-0b4840da5571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:49.790307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afda768-ec04-4ff9-a8e4-b4ac4e8691e3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:49.828206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e4278-1b0e-451b-9450-d03136484863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:49.829654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e4278-1b0e-451b-9450-d03136484863 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:49.830173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e4278-1b0e-451b-9450-d03136484863 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:49.830671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e4278-1b0e-451b-9450-d03136484863 None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:49.832064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e4278-1b0e-451b-9450-d03136484863 None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:49.833940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e4278-1b0e-451b-9450-d03136484863 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:49.836129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 195/386] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:49 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:49.850086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e4278-1b0e-451b-9450-d03136484863 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:49.867063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:49.881107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 192/387] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:49.903556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c02cb-b596-421f-838c-0b4840da5571 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:49.989094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e4278-1b0e-451b-9450-d03136484863 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:50.022192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:50.041780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b8725-a57e-4363-b841-cf48d8be8e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:50.042580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b8725-a57e-4363-b841-cf48d8be8e89 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.043123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b8725-a57e-4363-b841-cf48d8be8e89 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.043402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b8725-a57e-4363-b841-cf48d8be8e89 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.044787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b8725-a57e-4363-b841-cf48d8be8e89 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:50.046994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b8725-a57e-4363-b841-cf48d8be8e89 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:50.049535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 196/388] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:49 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:50.050816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dcecd-e7ae-45de-86ba-7429d4e70205 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.051527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dcecd-e7ae-45de-86ba-7429d4e70205 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.051818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dcecd-e7ae-45de-86ba-7429d4e70205 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.053324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 197/389] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:50.054595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dcecd-e7ae-45de-86ba-7429d4e70205 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:50.087752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c02cb-b596-421f-838c-0b4840da5571 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:50.125873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20768319-2824-41bf-8067-e192794e2a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:50.126757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20768319-2824-41bf-8067-e192794e2a68 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.127041 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20768319-2824-41bf-8067-e192794e2a68 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.127314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20768319-2824-41bf-8067-e192794e2a68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.138294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20768319-2824-41bf-8067-e192794e2a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:50.139087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-20768319-2824-41bf-8067-e192794e2a68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:50.149768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:50.150858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:50.151751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 193/390] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:50.156649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c4968-167e-4f08-95f7-6f7bb27bfbe8 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.156649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c4968-167e-4f08-95f7-6f7bb27bfbe8 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.156649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c4968-167e-4f08-95f7-6f7bb27bfbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.171030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-545c4968-167e-4f08-95f7-6f7bb27bfbe8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:50.172800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-545c4968-167e-4f08-95f7-6f7bb27bfbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:50.173843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dcecd-e7ae-45de-86ba-7429d4e70205 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:50.215004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccdcf6-3a41-45ce-b56c-3a99f6b11a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:50.216419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccdcf6-3a41-45ce-b56c-3a99f6b11a7c None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.216736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccdcf6-3a41-45ce-b56c-3a99f6b11a7c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.217012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccdcf6-3a41-45ce-b56c-3a99f6b11a7c None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/groups/7c061376f4434aab8c840883a46f2fe1/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.219928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccdcf6-3a41-45ce-b56c-3a99f6b11a7c None admin] RBAC: Authorizing `identity:create_grant(project_id=129c32efbe2e4cc19ffc72afb96752e8, group_id=7c061376f4434aab8c840883a46f2fe1, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:50.222189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccdcf6-3a41-45ce-b56c-3a99f6b11a7c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:50.243329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 198/391] 10.222.0.47 () {58 vars in 1442 bytes} [Fri Jun 24 23:35:50 2022] PUT /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/groups/7c061376f4434aab8c840883a46f2fe1/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:35:50.244421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccdcf6-3a41-45ce-b56c-3a99f6b11a7c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:50.250318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 194/392] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:50.277145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c4968-167e-4f08-95f7-6f7bb27bfbe8 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:50.333485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:50.407228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c4968-167e-4f08-95f7-6f7bb27bfbe8 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:50.433165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccdcf6-3a41-45ce-b56c-3a99f6b11a7c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:50.452151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df8b86-881a-4b49-9bbb-8c3fe7e05941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:50.453263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df8b86-881a-4b49-9bbb-8c3fe7e05941 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.454067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df8b86-881a-4b49-9bbb-8c3fe7e05941 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.454345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df8b86-881a-4b49-9bbb-8c3fe7e05941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.467841 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df8b86-881a-4b49-9bbb-8c3fe7e05941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:50.476371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139be339-e0ef-414e-a034-9bc478d16c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:50.477082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139be339-e0ef-414e-a034-9bc478d16c87 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.477371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139be339-e0ef-414e-a034-9bc478d16c87 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-01df8b86-881a-4b49-9bbb-8c3fe7e05941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:50.477488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:50.478312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:50.479152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:50.479732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139be339-e0ef-414e-a034-9bc478d16c87 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.479732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139be339-e0ef-414e-a034-9bc478d16c87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:50.479732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139be339-e0ef-414e-a034-9bc478d16c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:50.480415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 195/393] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:50.480613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139be339-e0ef-414e-a034-9bc478d16c87 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:50.483176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:50.486385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df8b86-881a-4b49-9bbb-8c3fe7e05941 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:50.493412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 199/394] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:50.499320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139be339-e0ef-414e-a034-9bc478d16c87 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:50.651274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df8b86-881a-4b49-9bbb-8c3fe7e05941 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:50.654968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:50.670482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139be339-e0ef-414e-a034-9bc478d16c87 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:50.701286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e918a42-ec34-472b-bb2c-6a440538ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:50.701968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e918a42-ec34-472b-bb2c-6a440538ce97 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.702255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e918a42-ec34-472b-bb2c-6a440538ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.702534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e918a42-ec34-472b-bb2c-6a440538ce97 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.703291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e918a42-ec34-472b-bb2c-6a440538ce97 None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:50.703664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e918a42-ec34-472b-bb2c-6a440538ce97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:50.705271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e918a42-ec34-472b-bb2c-6a440538ce97 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:50.720971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 196/395] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:50.726356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e918a42-ec34-472b-bb2c-6a440538ce97 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:50.735480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643bafc-686e-49b5-b276-5a6fb5cde0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:50.736445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643bafc-686e-49b5-b276-5a6fb5cde0ff None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.736749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643bafc-686e-49b5-b276-5a6fb5cde0ff None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.737025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643bafc-686e-49b5-b276-5a6fb5cde0ff None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.742928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643bafc-686e-49b5-b276-5a6fb5cde0ff None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:50.744390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643bafc-686e-49b5-b276-5a6fb5cde0ff None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:50.746069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 200/396] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:50 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:50.751974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643bafc-686e-49b5-b276-5a6fb5cde0ff None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:50.864960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:50.914230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e918a42-ec34-472b-bb2c-6a440538ce97 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:50.920438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643bafc-686e-49b5-b276-5a6fb5cde0ff None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:50.952251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe7340-3cc5-47d7-851a-b4cdefdd4ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:50.953145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe7340-3cc5-47d7-851a-b4cdefdd4ade None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.953427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe7340-3cc5-47d7-851a-b4cdefdd4ade None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.954208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe7340-3cc5-47d7-851a-b4cdefdd4ade None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.955820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe7340-3cc5-47d7-851a-b4cdefdd4ade None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:50.957341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe7340-3cc5-47d7-851a-b4cdefdd4ade None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:50.959016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 197/397] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:50 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:50.970693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe7340-3cc5-47d7-851a-b4cdefdd4ade None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:50.990552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c0e7a-23ef-4707-9599-2178d1c3d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:50.991438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c0e7a-23ef-4707-9599-2178d1c3d861 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.991723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c0e7a-23ef-4707-9599-2178d1c3d861 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.992002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c0e7a-23ef-4707-9599-2178d1c3d861 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:50.993244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c0e7a-23ef-4707-9599-2178d1c3d861 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:50.994684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c0e7a-23ef-4707-9599-2178d1c3d861 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:50.996735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 201/398] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:50 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:50.997991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903bbac-d7f5-4f6e-bdc2-b6cb6cc70f59 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:50.998281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903bbac-d7f5-4f6e-bdc2-b6cb6cc70f59 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:50.998562 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903bbac-d7f5-4f6e-bdc2-b6cb6cc70f59 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.000051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 202/399] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:51.004317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903bbac-d7f5-4f6e-bdc2-b6cb6cc70f59 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:51.017241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:51.103664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe7340-3cc5-47d7-851a-b4cdefdd4ade None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:51.125700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903bbac-d7f5-4f6e-bdc2-b6cb6cc70f59 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:51.140641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1a585-596b-4c27-86f3-74e50ef00edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:51.142278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a585-596b-4c27-86f3-74e50ef00edb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.142508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a585-596b-4c27-86f3-74e50ef00edb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.142745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1a585-596b-4c27-86f3-74e50ef00edb None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.143995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1a585-596b-4c27-86f3-74e50ef00edb None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:51.145840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1a585-596b-4c27-86f3-74e50ef00edb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:51.147639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 198/400] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:50 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:51.152507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5198d-7ba9-4c40-b450-f12c265d2669 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.152507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5198d-7ba9-4c40-b450-f12c265d2669 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.152507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5198d-7ba9-4c40-b450-f12c265d2669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.168914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-93f5198d-7ba9-4c40-b450-f12c265d2669 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:51.170278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-93f5198d-7ba9-4c40-b450-f12c265d2669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:51.176600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa8084-04ca-4c31-983f-be96801609ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:51.177767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa8084-04ca-4c31-983f-be96801609ee None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.178266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa8084-04ca-4c31-983f-be96801609ee None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.178732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa8084-04ca-4c31-983f-be96801609ee None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/groups/7c061376f4434aab8c840883a46f2fe1/roles/32433ed911104bb68970c0a00d0ddd49` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.182032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa8084-04ca-4c31-983f-be96801609ee None admin] RBAC: Authorizing `identity:create_grant(project_id=129c32efbe2e4cc19ffc72afb96752e8, group_id=7c061376f4434aab8c840883a46f2fe1, role_id=32433ed911104bb68970c0a00d0ddd49)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:51.184378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa8084-04ca-4c31-983f-be96801609ee None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:51.187602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:51.205002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 203/401] 10.222.0.47 () {58 vars in 1442 bytes} [Fri Jun 24 23:35:51 2022] PUT /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8/groups/7c061376f4434aab8c840883a46f2fe1/roles/32433ed911104bb68970c0a00d0ddd49 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:51.206417 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa8084-04ca-4c31-983f-be96801609ee None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:51.269253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 199/402] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:51.299228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5198d-7ba9-4c40-b450-f12c265d2669 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:51.350614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa8084-04ca-4c31-983f-be96801609ee None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:51.358789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:51.391749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b41a7-8c7c-4556-aa66-0cdd16310bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:51.392714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b41a7-8c7c-4556-aa66-0cdd16310bfd None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.392998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b41a7-8c7c-4556-aa66-0cdd16310bfd None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.393276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b41a7-8c7c-4556-aa66-0cdd16310bfd None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de/groups/7c061376f4434aab8c840883a46f2fe1/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.395341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b41a7-8c7c-4556-aa66-0cdd16310bfd None admin] RBAC: Authorizing `identity:create_grant(project_id=9b25ac538fb448279a63db11f84f70de, group_id=7c061376f4434aab8c840883a46f2fe1, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:51.396830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b41a7-8c7c-4556-aa66-0cdd16310bfd None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:51.416005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 204/403] 10.222.0.47 () {58 vars in 1442 bytes} [Fri Jun 24 23:35:51 2022] PUT /identity/v3/projects/9b25ac538fb448279a63db11f84f70de/groups/7c061376f4434aab8c840883a46f2fe1/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:51.418065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa714-923c-4027-b1f2-f8bf4682f82a None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.418361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa714-923c-4027-b1f2-f8bf4682f82a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.418700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37aa714-923c-4027-b1f2-f8bf4682f82a None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.424820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 205/404] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:51.427919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c34a1-d247-4cc5-87c4-5504ddb0beab None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.428442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c34a1-d247-4cc5-87c4-5504ddb0beab None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.430254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c34a1-d247-4cc5-87c4-5504ddb0beab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.455259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8d1c34a1-d247-4cc5-87c4-5504ddb0beab None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:51.456696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5198d-7ba9-4c40-b450-f12c265d2669 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:51.457646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8d1c34a1-d247-4cc5-87c4-5504ddb0beab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:51.509048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d3608-5b96-4098-aff0-5168e188ccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:51.510287 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d3608-5b96-4098-aff0-5168e188ccb6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.510740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d3608-5b96-4098-aff0-5168e188ccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.511267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d3608-5b96-4098-aff0-5168e188ccb6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.526345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d3608-5b96-4098-aff0-5168e188ccb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:51.530958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-5c2d3608-5b96-4098-aff0-5168e188ccb6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:51.542010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:51.542948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:51.544056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:51.547347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 200/405] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:51.553741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d3608-5b96-4098-aff0-5168e188ccb6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:51.558525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 206/406] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:51.592121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c34a1-d247-4cc5-87c4-5504ddb0beab None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:51.664644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:51.677146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d3608-5b96-4098-aff0-5168e188ccb6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:51.709129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c34a1-d247-4cc5-87c4-5504ddb0beab None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:51.721870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e55577-4e5e-438f-a6f7-84e6136ceb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:51.723186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e55577-4e5e-438f-a6f7-84e6136ceb5a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.723893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e55577-4e5e-438f-a6f7-84e6136ceb5a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.724314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e55577-4e5e-438f-a6f7-84e6136ceb5a None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.725547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e55577-4e5e-438f-a6f7-84e6136ceb5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:51.726011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e55577-4e5e-438f-a6f7-84e6136ceb5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:51.728150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e55577-4e5e-438f-a6f7-84e6136ceb5a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:51.740212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 201/407] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:51.745859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e55577-4e5e-438f-a6f7-84e6136ceb5a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:51.750155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b06453-d30c-4cec-8440-244edbd132f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:51.751419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b06453-d30c-4cec-8440-244edbd132f9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.751820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b06453-d30c-4cec-8440-244edbd132f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.752249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b06453-d30c-4cec-8440-244edbd132f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.765448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b06453-d30c-4cec-8440-244edbd132f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-43b06453-d30c-4cec-8440-244edbd132f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:51.774283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:51.775167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:51.775938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:51.781161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 207/408] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:35:51.782328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b06453-d30c-4cec-8440-244edbd132f9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:51.865222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e55577-4e5e-438f-a6f7-84e6136ceb5a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:51.884658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:51.922603 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83783f-3292-42ef-a4cd-06b2f2293c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:51.923871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83783f-3292-42ef-a4cd-06b2f2293c10 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:51.924333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83783f-3292-42ef-a4cd-06b2f2293c10 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:51.924957 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83783f-3292-42ef-a4cd-06b2f2293c10 None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:51.926363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83783f-3292-42ef-a4cd-06b2f2293c10 None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:51.928028 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83783f-3292-42ef-a4cd-06b2f2293c10 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:51.930300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 202/409] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:51 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:51.937641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83783f-3292-42ef-a4cd-06b2f2293c10 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:51.973048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:52.057362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83783f-3292-42ef-a4cd-06b2f2293c10 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:52.080972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b06453-d30c-4cec-8440-244edbd132f9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:52.100732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b7285-f3e6-43b6-b2d6-40d2e0f3dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:52.101996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b7285-f3e6-43b6-b2d6-40d2e0f3dd65 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:52.102494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b7285-f3e6-43b6-b2d6-40d2e0f3dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:52.102995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b7285-f3e6-43b6-b2d6-40d2e0f3dd65 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:52.104412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b7285-f3e6-43b6-b2d6-40d2e0f3dd65 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:52.106480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b7285-f3e6-43b6-b2d6-40d2e0f3dd65 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:52.108734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 203/410] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:51 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:52.114032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b7285-f3e6-43b6-b2d6-40d2e0f3dd65 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:52.146428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691f2d5-fe2e-40e6-9a4a-c84487ee8496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:52.147106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691f2d5-fe2e-40e6-9a4a-c84487ee8496 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:52.147389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691f2d5-fe2e-40e6-9a4a-c84487ee8496 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:52.147662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691f2d5-fe2e-40e6-9a4a-c84487ee8496 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:52.157093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691f2d5-fe2e-40e6-9a4a-c84487ee8496 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:52.157093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691f2d5-fe2e-40e6-9a4a-c84487ee8496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:52.157093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691f2d5-fe2e-40e6-9a4a-c84487ee8496 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:52.165933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 208/411] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:52.173794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691f2d5-fe2e-40e6-9a4a-c84487ee8496 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:52.262730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:52.285981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b7285-f3e6-43b6-b2d6-40d2e0f3dd65 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:52.341643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691f2d5-fe2e-40e6-9a4a-c84487ee8496 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:52.350759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:52.370639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8679-b7d7-4540-9c02-856c702e5982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:52.371685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8679-b7d7-4540-9c02-856c702e5982 None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:52.371817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8679-b7d7-4540-9c02-856c702e5982 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:52.372073 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8679-b7d7-4540-9c02-856c702e5982 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de/groups/7c061376f4434aab8c840883a46f2fe1/roles/32433ed911104bb68970c0a00d0ddd49` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:52.382406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd8679-b7d7-4540-9c02-856c702e5982 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b25ac538fb448279a63db11f84f70de, group_id=7c061376f4434aab8c840883a46f2fe1, role_id=32433ed911104bb68970c0a00d0ddd49)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:52.384002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd8679-b7d7-4540-9c02-856c702e5982 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:52.384465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbf9d1-fc7d-4784-bde5-cf2ec3e914bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:52.385633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbf9d1-fc7d-4784-bde5-cf2ec3e914bd None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:52.386107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbf9d1-fc7d-4784-bde5-cf2ec3e914bd None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:52.386764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbf9d1-fc7d-4784-bde5-cf2ec3e914bd None admin] PATH_INFO: `/v3/groups/3f872f13c64448b389b1d9f4f8f34c41` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:52.393543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbf9d1-fc7d-4784-bde5-cf2ec3e914bd None admin] RBAC: Authorizing `identity:get_group(group_id=3f872f13c64448b389b1d9f4f8f34c41)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:52.396355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbf9d1-fc7d-4784-bde5-cf2ec3e914bd None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:52.406985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 209/412] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:52 2022] GET /identity/v3/groups/3f872f13c64448b389b1d9f4f8f34c41 => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:52.408505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbf9d1-fc7d-4784-bde5-cf2ec3e914bd None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:52.411976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 204/413] 10.222.0.47 () {58 vars in 1442 bytes} [Fri Jun 24 23:35:52 2022] PUT /identity/v3/projects/9b25ac538fb448279a63db11f84f70de/groups/7c061376f4434aab8c840883a46f2fe1/roles/32433ed911104bb68970c0a00d0ddd49 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:52.525872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbf9d1-fc7d-4784-bde5-cf2ec3e914bd None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:52.563306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d6166-8498-4d10-ac70-f1a75816d238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:52.564653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d6166-8498-4d10-ac70-f1a75816d238 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:52.565140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d6166-8498-4d10-ac70-f1a75816d238 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:52.565606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d6166-8498-4d10-ac70-f1a75816d238 None admin] PATH_INFO: `/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:52.567126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d6166-8498-4d10-ac70-f1a75816d238 None admin] RBAC: Authorizing `identity:get_project(project_id=1b6f1fb3ca4d42909a61ac0a3da08290)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:52.568846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d6166-8498-4d10-ac70-f1a75816d238 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:52.575773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 210/414] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:52 2022] GET /identity/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:52.577677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8679-b7d7-4540-9c02-856c702e5982 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:52.712068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8679-b7d7-4540-9c02-856c702e5982 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:52.749562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcdfc28-88c8-457a-aea1-4f8e1de47978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:52.750653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdfc28-88c8-457a-aea1-4f8e1de47978 None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:52.750943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdfc28-88c8-457a-aea1-4f8e1de47978 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:52.751225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcdfc28-88c8-457a-aea1-4f8e1de47978 None admin] PATH_INFO: `/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290/groups/3f872f13c64448b389b1d9f4f8f34c41/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:52.753258 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdfc28-88c8-457a-aea1-4f8e1de47978 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b6f1fb3ca4d42909a61ac0a3da08290, group_id=3f872f13c64448b389b1d9f4f8f34c41, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:52.754705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcdfc28-88c8-457a-aea1-4f8e1de47978 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:52.774908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c23c-26a4-4ec2-8215-ab05a8882d0c None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:52.775429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c23c-26a4-4ec2-8215-ab05a8882d0c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:52.775603 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 205/415] 10.222.0.47 () {58 vars in 1442 bytes} [Fri Jun 24 23:35:52 2022] PUT /identity/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290/groups/3f872f13c64448b389b1d9f4f8f34c41/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:35:52.775767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c23c-26a4-4ec2-8215-ab05a8882d0c None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:52.779387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 211/416] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:35:52.792320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3244f0-c994-4cbb-9d28-1cbddb517d3f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:52.793037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3244f0-c994-4cbb-9d28-1cbddb517d3f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:52.793545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3244f0-c994-4cbb-9d28-1cbddb517d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:52.821380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ea3244f0-c994-4cbb-9d28-1cbddb517d3f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:52.823063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ea3244f0-c994-4cbb-9d28-1cbddb517d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:52.825785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:52.906442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 206/417] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:52.950071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c23c-26a4-4ec2-8215-ab05a8882d0c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:53.004946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:53.066153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c23c-26a4-4ec2-8215-ab05a8882d0c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:53.100027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171c9f-230a-487b-878c-103eaec8ed70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:53.100847 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171c9f-230a-487b-878c-103eaec8ed70 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:53.101126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171c9f-230a-487b-878c-103eaec8ed70 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:53.101400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171c9f-230a-487b-878c-103eaec8ed70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:53.113199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a171c9f-230a-487b-878c-103eaec8ed70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-8a171c9f-230a-487b-878c-103eaec8ed70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:53.120858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:53.121734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:53.122499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 212/418] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:53.129962 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3244f0-c994-4cbb-9d28-1cbddb517d3f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:53.253775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3244f0-c994-4cbb-9d28-1cbddb517d3f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:53.294198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b17b2-6da8-41d7-a23a-d5b7376dbe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:53.294879 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b17b2-6da8-41d7-a23a-d5b7376dbe57 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:53.295166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b17b2-6da8-41d7-a23a-d5b7376dbe57 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:53.295440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b17b2-6da8-41d7-a23a-d5b7376dbe57 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:53.296194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b17b2-6da8-41d7-a23a-d5b7376dbe57 None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:53.296500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b17b2-6da8-41d7-a23a-d5b7376dbe57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:53.298708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b17b2-6da8-41d7-a23a-d5b7376dbe57 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:53.304413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:53.312390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 207/419] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:53.318629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171c9f-230a-487b-878c-103eaec8ed70 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:53.481449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171c9f-230a-487b-878c-103eaec8ed70 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:53.483562 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:53.525564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b2b30-b178-44dc-90db-77f6682082c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:53.526229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b2b30-b178-44dc-90db-77f6682082c6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:53.526515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b2b30-b178-44dc-90db-77f6682082c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:53.526792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b2b30-b178-44dc-90db-77f6682082c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:53.610740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b2b30-b178-44dc-90db-77f6682082c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:53.612218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b2b30-b178-44dc-90db-77f6682082c6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:53.736431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 213/420] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:53.745668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b17b2-6da8-41d7-a23a-d5b7376dbe57 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:53.942106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd9a53-24b8-4e6e-8580-241701eaf2a0 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:53.942455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd9a53-24b8-4e6e-8580-241701eaf2a0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:53.942633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd9a53-24b8-4e6e-8580-241701eaf2a0 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:53.944198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 214/421] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:53.949248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b882d09-42ad-4176-96a8-a0e98695da40 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:53.949536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b882d09-42ad-4176-96a8-a0e98695da40 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:53.949815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b882d09-42ad-4176-96a8-a0e98695da40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:53.955569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:53.967669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-6b882d09-42ad-4176-96a8-a0e98695da40 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:53.968920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6b882d09-42ad-4176-96a8-a0e98695da40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:54.009484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b17b2-6da8-41d7-a23a-d5b7376dbe57 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:54.057413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 215/422] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:54.059051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e171c73-fefe-4c60-b515-33fd00443d0c None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.059612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e171c73-fefe-4c60-b515-33fd00443d0c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.060079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e171c73-fefe-4c60-b515-33fd00443d0c None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.061567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 216/423] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:54.067366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179441d0-9a2b-4786-9535-39d88323a518 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.067881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179441d0-9a2b-4786-9535-39d88323a518 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.068371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179441d0-9a2b-4786-9535-39d88323a518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.076525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e992c-5664-44e3-a419-28f7bad12e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:54.077424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e992c-5664-44e3-a419-28f7bad12e06 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.077713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e992c-5664-44e3-a419-28f7bad12e06 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.077996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e992c-5664-44e3-a419-28f7bad12e06 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.082200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e992c-5664-44e3-a419-28f7bad12e06 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:54.083988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e992c-5664-44e3-a419-28f7bad12e06 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:54.086020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 208/424] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:53 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:54.087638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-179441d0-9a2b-4786-9535-39d88323a518 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:54.089039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-179441d0-9a2b-4786-9535-39d88323a518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:54.091517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e992c-5664-44e3-a419-28f7bad12e06 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:54.176603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 217/425] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:35:54.176603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179441d0-9a2b-4786-9535-39d88323a518 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:54.199077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:54.240572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e992c-5664-44e3-a419-28f7bad12e06 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:54.268374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:54.279387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c856b7-068d-4413-bf33-e6f3105095dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:54.280262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c856b7-068d-4413-bf33-e6f3105095dd None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.280538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c856b7-068d-4413-bf33-e6f3105095dd None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.280847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c856b7-068d-4413-bf33-e6f3105095dd None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.282631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c856b7-068d-4413-bf33-e6f3105095dd None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:54.284554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c856b7-068d-4413-bf33-e6f3105095dd None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:54.286148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 209/426] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:54 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:54.286932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c856b7-068d-4413-bf33-e6f3105095dd None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:54.322566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179441d0-9a2b-4786-9535-39d88323a518 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:54.365476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280731a3-3e0c-4541-8c6f-2939320d524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:54.366434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280731a3-3e0c-4541-8c6f-2939320d524e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.367212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280731a3-3e0c-4541-8c6f-2939320d524e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.367212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280731a3-3e0c-4541-8c6f-2939320d524e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.379250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280731a3-3e0c-4541-8c6f-2939320d524e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-280731a3-3e0c-4541-8c6f-2939320d524e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:54.387469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:54.388533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 23:35:54.389536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:54.396327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 218/427] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:54.397335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280731a3-3e0c-4541-8c6f-2939320d524e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:54.410989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c856b7-068d-4413-bf33-e6f3105095dd None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:54.457968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f646b-62db-4c7f-ab5a-b4854f423213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:54.459263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f646b-62db-4c7f-ab5a-b4854f423213 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.459690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f646b-62db-4c7f-ab5a-b4854f423213 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.460505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f646b-62db-4c7f-ab5a-b4854f423213 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.464468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:54.476409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f646b-62db-4c7f-ab5a-b4854f423213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-5f7f646b-62db-4c7f-ab5a-b4854f423213 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:54.487332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:54.488412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:54.489447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:54.497117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 210/428] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:54.498528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f646b-62db-4c7f-ab5a-b4854f423213 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:54.511338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280731a3-3e0c-4541-8c6f-2939320d524e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:54.537062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:54.551412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b106f-af1c-46bd-8220-f36c71651087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:54.552104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b106f-af1c-46bd-8220-f36c71651087 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.552446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b106f-af1c-46bd-8220-f36c71651087 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.552765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b106f-af1c-46bd-8220-f36c71651087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.553784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b106f-af1c-46bd-8220-f36c71651087 None admin] RBAC: Adding query filter params (group.id=7c061376f4434aab8c840883a46f2fe1, role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:54.554107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b106f-af1c-46bd-8220-f36c71651087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:54.555560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b106f-af1c-46bd-8220-f36c71651087 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:54.568764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 219/429] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:54 2022] GET /identity/v3/role_assignments?group.id=7c061376f4434aab8c840883a46f2fe1&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=86c49ec8587244e48fab81862ddb3641 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:54.569959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b106f-af1c-46bd-8220-f36c71651087 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:54.627049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f646b-62db-4c7f-ab5a-b4854f423213 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:54.667566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac40269-7dd6-49f7-8f47-51896d690a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:54.668654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac40269-7dd6-49f7-8f47-51896d690a5a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.669545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac40269-7dd6-49f7-8f47-51896d690a5a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.670186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac40269-7dd6-49f7-8f47-51896d690a5a None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.671161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac40269-7dd6-49f7-8f47-51896d690a5a None admin] RBAC: Adding query filter params (name=member) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:54.671673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac40269-7dd6-49f7-8f47-51896d690a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:54.674084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac40269-7dd6-49f7-8f47-51896d690a5a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:54.684507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 211/430] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:35:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:54.694968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac40269-7dd6-49f7-8f47-51896d690a5a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:54.770303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:54.820304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac40269-7dd6-49f7-8f47-51896d690a5a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:54.822591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b106f-af1c-46bd-8220-f36c71651087 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:54.858147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50918b-7f29-41c5-8961-a9db2d5d3f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:54.858882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50918b-7f29-41c5-8961-a9db2d5d3f12 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.859168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50918b-7f29-41c5-8961-a9db2d5d3f12 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.859449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50918b-7f29-41c5-8961-a9db2d5d3f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.860317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ad709-e06e-4551-a291-dce8c77b680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:54.861508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad709-e06e-4551-a291-dce8c77b680c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:54.861994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad709-e06e-4551-a291-dce8c77b680c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:54.862567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad709-e06e-4551-a291-dce8c77b680c None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:54.863529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ad709-e06e-4551-a291-dce8c77b680c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:54.864036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ad709-e06e-4551-a291-dce8c77b680c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:54.865670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ad709-e06e-4551-a291-dce8c77b680c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:54.884671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 220/431] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:54.888015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ad709-e06e-4551-a291-dce8c77b680c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:54.950201 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:54.968683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50918b-7f29-41c5-8961-a9db2d5d3f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:54.971272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50918b-7f29-41c5-8961-a9db2d5d3f12 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:55.007395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ad709-e06e-4551-a291-dce8c77b680c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.044271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:55.047193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc18bc9-591b-4936-b477-06c2b890928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:55.047789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc18bc9-591b-4936-b477-06c2b890928f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.048617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc18bc9-591b-4936-b477-06c2b890928f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.048617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc18bc9-591b-4936-b477-06c2b890928f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.098653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 212/432] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:55.099795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a64ebf-a985-46f3-9b35-56f238bb18ae None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.100558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a64ebf-a985-46f3-9b35-56f238bb18ae None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.101370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a64ebf-a985-46f3-9b35-56f238bb18ae None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.102928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 213/433] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:55.105302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a64ebf-a985-46f3-9b35-56f238bb18ae None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:55.139329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc18bc9-591b-4936-b477-06c2b890928f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:55.141028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc18bc9-591b-4936-b477-06c2b890928f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:55.206486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:55.227978 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a64ebf-a985-46f3-9b35-56f238bb18ae None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.270372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b61f8-c98d-46f9-ae60-1dfe8464daa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:55.271248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b61f8-c98d-46f9-ae60-1dfe8464daa7 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.271532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b61f8-c98d-46f9-ae60-1dfe8464daa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.271806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b61f8-c98d-46f9-ae60-1dfe8464daa7 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.273313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b61f8-c98d-46f9-ae60-1dfe8464daa7 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:55.274743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b61f8-c98d-46f9-ae60-1dfe8464daa7 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:55.276637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 214/434] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:55 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:55.277617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdde980-22b2-4e99-9eca-0e56e3583f9f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.277906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdde980-22b2-4e99-9eca-0e56e3583f9f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.278279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdde980-22b2-4e99-9eca-0e56e3583f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.288171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 221/435] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 23:35:55.293464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc18bc9-591b-4936-b477-06c2b890928f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:55.299419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-fbdde980-22b2-4e99-9eca-0e56e3583f9f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:55.300697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-fbdde980-22b2-4e99-9eca-0e56e3583f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:55.323199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:55.369867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 215/436] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:55.370703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdde980-22b2-4e99-9eca-0e56e3583f9f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:55.397710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc18bc9-591b-4936-b477-06c2b890928f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.432315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7a307-b524-427d-b72e-79f356e1c9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:55.433201 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7a307-b524-427d-b72e-79f356e1c9b7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.433481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7a307-b524-427d-b72e-79f356e1c9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.433755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7a307-b524-427d-b72e-79f356e1c9b7 None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.434913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7a307-b524-427d-b72e-79f356e1c9b7 None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:55.436366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7a307-b524-427d-b72e-79f356e1c9b7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:55.438161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 222/437] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:55 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:55.439314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75ed00-ed83-4aa2-b923-46dcb4f4dc7a None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.439599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75ed00-ed83-4aa2-b923-46dcb4f4dc7a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.439883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75ed00-ed83-4aa2-b923-46dcb4f4dc7a None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.441125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 223/438] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:55.441816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75ed00-ed83-4aa2-b923-46dcb4f4dc7a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:55.459607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:55.474854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdde980-22b2-4e99-9eca-0e56e3583f9f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.508677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844adad3-21dc-4a85-ae30-ad9e01ecca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:55.509538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844adad3-21dc-4a85-ae30-ad9e01ecca96 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.509821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844adad3-21dc-4a85-ae30-ad9e01ecca96 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.510093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844adad3-21dc-4a85-ae30-ad9e01ecca96 None admin] PATH_INFO: `/v3/projects/129c32efbe2e4cc19ffc72afb96752e8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.511277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844adad3-21dc-4a85-ae30-ad9e01ecca96 None admin] RBAC: Authorizing `identity:get_project(project_id=129c32efbe2e4cc19ffc72afb96752e8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:55.512749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844adad3-21dc-4a85-ae30-ad9e01ecca96 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:55.514425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 216/439] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:55 2022] GET /identity/v3/projects/129c32efbe2e4cc19ffc72afb96752e8 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:55.515181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844adad3-21dc-4a85-ae30-ad9e01ecca96 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:55.548563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75ed00-ed83-4aa2-b923-46dcb4f4dc7a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.551845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:55.584468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db4c25-6d13-425f-a2f8-1c3e7c142bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:55.585307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db4c25-6d13-425f-a2f8-1c3e7c142bd9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.585592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db4c25-6d13-425f-a2f8-1c3e7c142bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.585864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db4c25-6d13-425f-a2f8-1c3e7c142bd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.596000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db4c25-6d13-425f-a2f8-1c3e7c142bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-a1db4c25-6d13-425f-a2f8-1c3e7c142bd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:55.604177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:55.605116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:35:55.605893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 224/440] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:35:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:55.606688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d39307-6c67-453a-b98a-4a5a565fe512 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.607058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d39307-6c67-453a-b98a-4a5a565fe512 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.607167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d39307-6c67-453a-b98a-4a5a565fe512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.623805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-90d39307-6c67-453a-b98a-4a5a565fe512 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:55.625040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-90d39307-6c67-453a-b98a-4a5a565fe512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:55.631250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844adad3-21dc-4a85-ae30-ad9e01ecca96 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.666225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4503d-b426-4441-a88c-329e483c3467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:55.666897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4503d-b426-4441-a88c-329e483c3467 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.667181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4503d-b426-4441-a88c-329e483c3467 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.667456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4503d-b426-4441-a88c-329e483c3467 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.668279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4503d-b426-4441-a88c-329e483c3467 None admin] RBAC: Adding query filter params (group.id=7c061376f4434aab8c840883a46f2fe1, role.id=86c49ec8587244e48fab81862ddb3641, scope.project.id=9b25ac538fb448279a63db11f84f70de) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:55.668612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4503d-b426-4441-a88c-329e483c3467 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:55.670042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4503d-b426-4441-a88c-329e483c3467 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:55.675726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 217/441] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:55 2022] GET /identity/v3/role_assignments?group.id=7c061376f4434aab8c840883a46f2fe1&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=86c49ec8587244e48fab81862ddb3641 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:55.675726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4503d-b426-4441-a88c-329e483c3467 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:55.700081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 225/442] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:55.700081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d39307-6c67-453a-b98a-4a5a565fe512 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:55.702555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:55.800649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:55.814746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d39307-6c67-453a-b98a-4a5a565fe512 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.815610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4503d-b426-4441-a88c-329e483c3467 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.854083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1836d-a0d3-429c-81a8-08d6b05e47bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:55.854951 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1836d-a0d3-429c-81a8-08d6b05e47bd None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.855225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1836d-a0d3-429c-81a8-08d6b05e47bd None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.855502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1836d-a0d3-429c-81a8-08d6b05e47bd None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.856790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1836d-a0d3-429c-81a8-08d6b05e47bd None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:55.856913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dbef0-c687-4826-a351-ee2201b9a9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:55.857493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dbef0-c687-4826-a351-ee2201b9a9ba None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.857772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dbef0-c687-4826-a351-ee2201b9a9ba None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:55.858044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dbef0-c687-4826-a351-ee2201b9a9ba None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:55.858522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1836d-a0d3-429c-81a8-08d6b05e47bd None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:55.858790 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009dbef0-c687-4826-a351-ee2201b9a9ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:55.859094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009dbef0-c687-4826-a351-ee2201b9a9ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:55.860290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 218/443] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:55 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:55.865437 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1836d-a0d3-429c-81a8-08d6b05e47bd None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:55.865577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009dbef0-c687-4826-a351-ee2201b9a9ba None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:55.870688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 226/444] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:35:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:55.871520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dbef0-c687-4826-a351-ee2201b9a9ba None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:55.944162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:55.964889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1836d-a0d3-429c-81a8-08d6b05e47bd None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.980728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dbef0-c687-4826-a351-ee2201b9a9ba None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:55.998861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0e03a-8e38-4ad4-83d5-f781565297f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:55.999652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0e03a-8e38-4ad4-83d5-f781565297f2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:55.999936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0e03a-8e38-4ad4-83d5-f781565297f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.000759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0e03a-8e38-4ad4-83d5-f781565297f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.010948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0e03a-8e38-4ad4-83d5-f781565297f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:35:56.013566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf351fa-fdd9-4d48-bafb-4a4d11cedbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:56.014221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf351fa-fdd9-4d48-bafb-4a4d11cedbd3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:56.014504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf351fa-fdd9-4d48-bafb-4a4d11cedbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.014777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf351fa-fdd9-4d48-bafb-4a4d11cedbd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.015603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf351fa-fdd9-4d48-bafb-4a4d11cedbd3 None admin] RBAC: Adding query filter params (group.id=7c061376f4434aab8c840883a46f2fe1, role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=129c32efbe2e4cc19ffc72afb96752e8) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:56.016619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf351fa-fdd9-4d48-bafb-4a4d11cedbd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:56.018061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf351fa-fdd9-4d48-bafb-4a4d11cedbd3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-fce0e03a-8e38-4ad4-83d5-f781565297f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:35:56.019029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:35:56.019898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:35:56.020726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 219/445] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:35:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:35:56.021374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0e03a-8e38-4ad4-83d5-f781565297f2 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:56.028446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 227/446] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:55 2022] GET /identity/v3/role_assignments?group.id=7c061376f4434aab8c840883a46f2fe1&scope.project.id=129c32efbe2e4cc19ffc72afb96752e8&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:56.029292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf351fa-fdd9-4d48-bafb-4a4d11cedbd3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:56.040150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:56.136603 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0e03a-8e38-4ad4-83d5-f781565297f2 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:56.152760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf351fa-fdd9-4d48-bafb-4a4d11cedbd3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:56.175788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540db7c-ba5d-4641-83ab-c21332aadb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:56.176559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540db7c-ba5d-4641-83ab-c21332aadb07 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:56.176873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540db7c-ba5d-4641-83ab-c21332aadb07 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.177145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540db7c-ba5d-4641-83ab-c21332aadb07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.187261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:56.194706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eddf98-69cc-4123-a58d-34640e22e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:56.195381 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eddf98-69cc-4123-a58d-34640e22e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:56.195662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eddf98-69cc-4123-a58d-34640e22e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.195940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eddf98-69cc-4123-a58d-34640e22e0f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.196730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eddf98-69cc-4123-a58d-34640e22e0f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:56.197328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eddf98-69cc-4123-a58d-34640e22e0f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:56.198395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eddf98-69cc-4123-a58d-34640e22e0f6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:56.211327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 228/447] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:35:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:35:56.216832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eddf98-69cc-4123-a58d-34640e22e0f6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:56.262722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540db7c-ba5d-4641-83ab-c21332aadb07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:56.264924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540db7c-ba5d-4641-83ab-c21332aadb07 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:56.285372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:56.330400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eddf98-69cc-4123-a58d-34640e22e0f6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:56.367221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa8ffd-9a15-4c7c-98b3-6f67e2f63c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:56.367892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa8ffd-9a15-4c7c-98b3-6f67e2f63c29 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:56.368179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa8ffd-9a15-4c7c-98b3-6f67e2f63c29 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.368465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa8ffd-9a15-4c7c-98b3-6f67e2f63c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.380679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 220/448] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:35:56.386361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540db7c-ba5d-4641-83ab-c21332aadb07 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:56.439938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:56.450350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa8ffd-9a15-4c7c-98b3-6f67e2f63c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:56.451999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa8ffd-9a15-4c7c-98b3-6f67e2f63c29 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:56.505747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540db7c-ba5d-4641-83ab-c21332aadb07 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:56.526796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:56.549355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37a2fb-2ba1-4613-927f-f3f2dc3b1b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:56.550225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37a2fb-2ba1-4613-927f-f3f2dc3b1b37 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:56.550510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37a2fb-2ba1-4613-927f-f3f2dc3b1b37 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.550780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37a2fb-2ba1-4613-927f-f3f2dc3b1b37 None admin] PATH_INFO: `/v3/projects/9b25ac538fb448279a63db11f84f70de` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.551968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37a2fb-2ba1-4613-927f-f3f2dc3b1b37 None admin] RBAC: Authorizing `identity:get_project(project_id=9b25ac538fb448279a63db11f84f70de)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:56.553470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37a2fb-2ba1-4613-927f-f3f2dc3b1b37 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:56.555155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 221/449] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:56 2022] GET /identity/v3/projects/9b25ac538fb448279a63db11f84f70de => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:56.560683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37a2fb-2ba1-4613-927f-f3f2dc3b1b37 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:56.561845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 229/450] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:35:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 23:35:56.567606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa8ffd-9a15-4c7c-98b3-6f67e2f63c29 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:56.659521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37a2fb-2ba1-4613-927f-f3f2dc3b1b37 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:56.659719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:56.667263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa8ffd-9a15-4c7c-98b3-6f67e2f63c29 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:56.694068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6788e-ec99-46f2-a6cb-41becf8e3d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:56.694854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6788e-ec99-46f2-a6cb-41becf8e3d88 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:56.695137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6788e-ec99-46f2-a6cb-41becf8e3d88 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.695420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6788e-ec99-46f2-a6cb-41becf8e3d88 None admin] PATH_INFO: `/v3/groups/7c061376f4434aab8c840883a46f2fe1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.696589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6788e-ec99-46f2-a6cb-41becf8e3d88 None admin] RBAC: Authorizing `identity:get_group(group_id=7c061376f4434aab8c840883a46f2fe1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:56.698078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6788e-ec99-46f2-a6cb-41becf8e3d88 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:56.704636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 222/451] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:56 2022] GET /identity/v3/groups/7c061376f4434aab8c840883a46f2fe1 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:56.704764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fcbc5-d1bf-4061-a56f-efc09e6109f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:56.704764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fcbc5-d1bf-4061-a56f-efc09e6109f2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:56.704764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fcbc5-d1bf-4061-a56f-efc09e6109f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.704764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fcbc5-d1bf-4061-a56f-efc09e6109f2 None admin] PATH_INFO: `/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.705091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fcbc5-d1bf-4061-a56f-efc09e6109f2 None admin] RBAC: Authorizing `identity:get_project(project_id=1b6f1fb3ca4d42909a61ac0a3da08290)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:56.705091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fcbc5-d1bf-4061-a56f-efc09e6109f2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:56.708687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 230/452] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:35:56 2022] GET /identity/v3/projects/1b6f1fb3ca4d42909a61ac0a3da08290 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:56.708809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6788e-ec99-46f2-a6cb-41becf8e3d88 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:56.711100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fcbc5-d1bf-4061-a56f-efc09e6109f2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:56.760795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:56.807051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6788e-ec99-46f2-a6cb-41becf8e3d88 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:56.813247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fcbc5-d1bf-4061-a56f-efc09e6109f2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:56.841956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75834e79-7d0a-4f59-af15-fff8740c52a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:56.842604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75834e79-7d0a-4f59-af15-fff8740c52a3 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:56.842887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75834e79-7d0a-4f59-af15-fff8740c52a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.843167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75834e79-7d0a-4f59-af15-fff8740c52a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.844144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75834e79-7d0a-4f59-af15-fff8740c52a3 None admin] RBAC: Adding query filter params (group.id=7c061376f4434aab8c840883a46f2fe1, role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=9b25ac538fb448279a63db11f84f70de) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:56.844460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75834e79-7d0a-4f59-af15-fff8740c52a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:56.845937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75834e79-7d0a-4f59-af15-fff8740c52a3 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:56.846323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520471a-20ac-4f08-bc11-d30f013bb35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:56.847174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520471a-20ac-4f08-bc11-d30f013bb35d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:56.847455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520471a-20ac-4f08-bc11-d30f013bb35d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:56.847739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520471a-20ac-4f08-bc11-d30f013bb35d None admin] PATH_INFO: `/v3/groups/3f872f13c64448b389b1d9f4f8f34c41` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:56.848954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520471a-20ac-4f08-bc11-d30f013bb35d None admin] RBAC: Authorizing `identity:get_group(group_id=3f872f13c64448b389b1d9f4f8f34c41)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:56.850372 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520471a-20ac-4f08-bc11-d30f013bb35d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:56.851678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 223/453] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:56 2022] GET /identity/v3/role_assignments?group.id=7c061376f4434aab8c840883a46f2fe1&scope.project.id=9b25ac538fb448279a63db11f84f70de&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:56.851971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 231/454] 10.222.0.47 () {56 vars in 1180 bytes} [Fri Jun 24 23:35:56 2022] GET /identity/v3/groups/3f872f13c64448b389b1d9f4f8f34c41 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:56.857720 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75834e79-7d0a-4f59-af15-fff8740c52a3 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:56.971733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75834e79-7d0a-4f59-af15-fff8740c52a3 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:56.989900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:57.010194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e09f-5a67-4f0b-a87b-e34196cee613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:57.010848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e09f-5a67-4f0b-a87b-e34196cee613 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:57.011131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e09f-5a67-4f0b-a87b-e34196cee613 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:57.011411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e09f-5a67-4f0b-a87b-e34196cee613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:57.012249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4e09f-5a67-4f0b-a87b-e34196cee613 None admin] RBAC: Adding query filter params (group.id=3f872f13c64448b389b1d9f4f8f34c41, role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=1b6f1fb3ca4d42909a61ac0a3da08290) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:35:57.012560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4e09f-5a67-4f0b-a87b-e34196cee613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:57.014021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf4e09f-5a67-4f0b-a87b-e34196cee613 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:35:57.021190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 224/455] 10.222.0.47 () {56 vars in 1376 bytes} [Fri Jun 24 23:35:56 2022] GET /identity/v3/role_assignments?group.id=3f872f13c64448b389b1d9f4f8f34c41&scope.project.id=1b6f1fb3ca4d42909a61ac0a3da08290&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:35:59.585860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8659b-1d5f-4937-882b-f1d535cd07b7 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:59.586407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8659b-1d5f-4937-882b-f1d535cd07b7 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:59.586407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8659b-1d5f-4937-882b-f1d535cd07b7 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:59.588001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 232/456] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:59.593711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d9ebb-537b-4a9a-ba70-7adf0264c1d4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:59.594715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d9ebb-537b-4a9a-ba70-7adf0264c1d4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:59.595216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d9ebb-537b-4a9a-ba70-7adf0264c1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:59.621616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ee3d9ebb-537b-4a9a-ba70-7adf0264c1d4 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:59.623236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ee3d9ebb-537b-4a9a-ba70-7adf0264c1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:59.719791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 225/457] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:59.723558 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71739e93-cb8b-46de-bed7-9f51a940ad00 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:59.724061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71739e93-cb8b-46de-bed7-9f51a940ad00 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:59.724601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71739e93-cb8b-46de-bed7-9f51a940ad00 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:59.726194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 233/458] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:59.731456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc8d86-40cb-49de-a3eb-3d603a1efb2e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:59.731738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc8d86-40cb-49de-a3eb-3d603a1efb2e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:59.732022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc8d86-40cb-49de-a3eb-3d603a1efb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:59.757680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-51fc8d86-40cb-49de-a3eb-3d603a1efb2e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:59.757971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71739e93-cb8b-46de-bed7-9f51a940ad00 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:35:59.758891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-51fc8d86-40cb-49de-a3eb-3d603a1efb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:59.813445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:59.884543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 226/459] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:35:59.893260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960fbb-a4a3-418f-ba7d-adee66fc504c None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:59.893559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960fbb-a4a3-418f-ba7d-adee66fc504c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:59.893843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b960fbb-a4a3-418f-ba7d-adee66fc504c None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:59.895370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 227/460] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:35:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:35:59.901913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40932200-b020-4c8c-aa9a-575b9302acdb None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:59.902200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40932200-b020-4c8c-aa9a-575b9302acdb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:59.902478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40932200-b020-4c8c-aa9a-575b9302acdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:59.924785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71739e93-cb8b-46de-bed7-9f51a940ad00 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:35:59.925521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-40932200-b020-4c8c-aa9a-575b9302acdb None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:35:59.927094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-40932200-b020-4c8c-aa9a-575b9302acdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:35:59.965805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:35:59.970620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbbd29-bb7c-43a2-8d16-f7e7a77e8dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:35:59.971370 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbbd29-bb7c-43a2-8d16-f7e7a77e8dd8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:35:59.971651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbbd29-bb7c-43a2-8d16-f7e7a77e8dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:35:59.971922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbbd29-bb7c-43a2-8d16-f7e7a77e8dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:35:59.984764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbbd29-bb7c-43a2-8d16-f7e7a77e8dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:35:59.987276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbbd29-bb7c-43a2-8d16-f7e7a77e8dd8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:00.006735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 228/461] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:00.008519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40932200-b020-4c8c-aa9a-575b9302acdb None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-84cbbd29-bb7c-43a2-8d16-f7e7a77e8dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:00.016880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:00.018434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:00.019721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:00.019721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:00.019721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:00.019721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:00.019721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 234/462] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:00.025325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbbd29-bb7c-43a2-8d16-f7e7a77e8dd8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:00.142836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:00.146194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40932200-b020-4c8c-aa9a-575b9302acdb None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:00.166364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbbd29-bb7c-43a2-8d16-f7e7a77e8dd8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:00.188776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc333a-279f-4fa7-b270-62b0c3435a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:00.189578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc333a-279f-4fa7-b270-62b0c3435a1c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.189863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc333a-279f-4fa7-b270-62b0c3435a1c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.190140 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc333a-279f-4fa7-b270-62b0c3435a1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.205170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc333a-279f-4fa7-b270-62b0c3435a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:00.206336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72384932-f8f0-4d91-916a-bfc15a70ec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:00.206800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc333a-279f-4fa7-b270-62b0c3435a1c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:00.207637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72384932-f8f0-4d91-916a-bfc15a70ec2e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.208120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72384932-f8f0-4d91-916a-bfc15a70ec2e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.208604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72384932-f8f0-4d91-916a-bfc15a70ec2e None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.209518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72384932-f8f0-4d91-916a-bfc15a70ec2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:00.210012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72384932-f8f0-4d91-916a-bfc15a70ec2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:00.211536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72384932-f8f0-4d91-916a-bfc15a70ec2e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:00.225097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 235/463] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:00.226204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72384932-f8f0-4d91-916a-bfc15a70ec2e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-0bcc333a-279f-4fa7-b270-62b0c3435a1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:00.228191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:00.228996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:00.229903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:00.229903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:00.229903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:00.229903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:00.232659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 229/464] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:00.233664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc333a-279f-4fa7-b270-62b0c3435a1c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:00.260291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:00.340352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc333a-279f-4fa7-b270-62b0c3435a1c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:00.340963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72384932-f8f0-4d91-916a-bfc15a70ec2e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:00.376470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52d55c-887b-461d-a5e6-1ad291994a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:00.377289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d55c-887b-461d-a5e6-1ad291994a71 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.377573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d55c-887b-461d-a5e6-1ad291994a71 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.377668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf9463-7893-46e4-a401-3808eac8e8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:00.377851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d55c-887b-461d-a5e6-1ad291994a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.378241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf9463-7893-46e4-a401-3808eac8e8f3 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.378528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf9463-7893-46e4-a401-3808eac8e8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.378805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf9463-7893-46e4-a401-3808eac8e8f3 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.379503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf9463-7893-46e4-a401-3808eac8e8f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:00.381060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf9463-7893-46e4-a401-3808eac8e8f3 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:00.388612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52d55c-887b-461d-a5e6-1ad291994a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:00.390034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52d55c-887b-461d-a5e6-1ad291994a71 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:00.393834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:00.404453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 230/465] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:36:00 2022] POST /identity/v3/users => generated 248 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:00.405461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf9463-7893-46e4-a401-3808eac8e8f3 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-8d52d55c-887b-461d-a5e6-1ad291994a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:00.412114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:00.413089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:00.413912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:00.413912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:00.413912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:00.413912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:00.413912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 236/466] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:00.415007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd003b82-8011-4b40-a6ca-54995a934333 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.415293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd003b82-8011-4b40-a6ca-54995a934333 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.415569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd003b82-8011-4b40-a6ca-54995a934333 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.421103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 237/467] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:00.421836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd003b82-8011-4b40-a6ca-54995a934333 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:00.505081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:00.533522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf9463-7893-46e4-a401-3808eac8e8f3 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:00.565751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd003b82-8011-4b40-a6ca-54995a934333 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:00.580610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd388c-349b-4983-8c00-3fb3b44e81bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:00.581900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd388c-349b-4983-8c00-3fb3b44e81bb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.582379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd388c-349b-4983-8c00-3fb3b44e81bb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.583096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd388c-349b-4983-8c00-3fb3b44e81bb None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.584529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd388c-349b-4983-8c00-3fb3b44e81bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:00.585005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd388c-349b-4983-8c00-3fb3b44e81bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:00.587614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd388c-349b-4983-8c00-3fb3b44e81bb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:00.601169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 231/468] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:00.602298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671689a4-ce15-4349-a0cb-df37799347f5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.602581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671689a4-ce15-4349-a0cb-df37799347f5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.602858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671689a4-ce15-4349-a0cb-df37799347f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.613100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5969e92-c940-4a04-9a1a-4a3b6ac7fde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:00.613773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5969e92-c940-4a04-9a1a-4a3b6ac7fde3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.614055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5969e92-c940-4a04-9a1a-4a3b6ac7fde3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.614333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5969e92-c940-4a04-9a1a-4a3b6ac7fde3 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.615069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5969e92-c940-4a04-9a1a-4a3b6ac7fde3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:00.615406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5969e92-c940-4a04-9a1a-4a3b6ac7fde3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:00.618023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5969e92-c940-4a04-9a1a-4a3b6ac7fde3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:00.624093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-671689a4-ce15-4349-a0cb-df37799347f5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:00.626162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-671689a4-ce15-4349-a0cb-df37799347f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:00.631613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 238/469] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:00.636208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5969e92-c940-4a04-9a1a-4a3b6ac7fde3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:00.705779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:00.711419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 232/470] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:00.712670 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671689a4-ce15-4349-a0cb-df37799347f5 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:00.767810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5969e92-c940-4a04-9a1a-4a3b6ac7fde3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:00.783878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:00.821435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf57aa-4071-4cb2-92b9-d667b9786d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:00.822148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf57aa-4071-4cb2-92b9-d667b9786d8d None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.822435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf57aa-4071-4cb2-92b9-d667b9786d8d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.822662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671689a4-ce15-4349-a0cb-df37799347f5 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:00.822794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf57aa-4071-4cb2-92b9-d667b9786d8d None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.823419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf57aa-4071-4cb2-92b9-d667b9786d8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:00.826056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf57aa-4071-4cb2-92b9-d667b9786d8d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:00.853976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 239/471] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:36:00 2022] POST /identity/v3/users => generated 246 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:00.855024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf57aa-4071-4cb2-92b9-d667b9786d8d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:00.873700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86f375-e04e-4adf-95aa-b02f4fba8382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:00.874442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86f375-e04e-4adf-95aa-b02f4fba8382 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:00.874734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86f375-e04e-4adf-95aa-b02f4fba8382 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:00.875205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86f375-e04e-4adf-95aa-b02f4fba8382 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:00.875946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86f375-e04e-4adf-95aa-b02f4fba8382 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:00.877558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86f375-e04e-4adf-95aa-b02f4fba8382 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:00.936173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 233/472] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:36:00 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:00.988035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf57aa-4071-4cb2-92b9-d667b9786d8d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:01.001004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:01.030874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8aed4-25bc-4a37-905c-a5a91587cb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:01.032874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8aed4-25bc-4a37-905c-a5a91587cb61 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:01.033159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8aed4-25bc-4a37-905c-a5a91587cb61 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:01.033677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8aed4-25bc-4a37-905c-a5a91587cb61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:01.040341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed03bf-abf6-48f7-9df6-ba2c02de099f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:01.040656 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed03bf-abf6-48f7-9df6-ba2c02de099f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:01.040936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed03bf-abf6-48f7-9df6-ba2c02de099f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:01.043827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 234/473] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:01.046555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8aed4-25bc-4a37-905c-a5a91587cb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:01.048500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8aed4-25bc-4a37-905c-a5a91587cb61 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:01.058569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8445f-87a5-4286-a3ec-5ba0d02656c2 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:01.059212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8445f-87a5-4286-a3ec-5ba0d02656c2 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:01.059687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8445f-87a5-4286-a3ec-5ba0d02656c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ffe8aed4-25bc-4a37-905c-a5a91587cb61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:01.071685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:01.072914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:01.073739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:01.073739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:01.073739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:01.073739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:01.073739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 240/474] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:01.089764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8aed4-25bc-4a37-905c-a5a91587cb61 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:01.095765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-85b8445f-87a5-4286-a3ec-5ba0d02656c2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:01.097384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-85b8445f-87a5-4286-a3ec-5ba0d02656c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:01.204760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 235/475] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:01.229414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8445f-87a5-4286-a3ec-5ba0d02656c2 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:01.229577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8aed4-25bc-4a37-905c-a5a91587cb61 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:01.308972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:01.318732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e52d-3c01-4bb7-9275-7d26a7bc5cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:01.319418 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e52d-3c01-4bb7-9275-7d26a7bc5cc8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:01.319733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e52d-3c01-4bb7-9275-7d26a7bc5cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:01.320009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781e52d-3c01-4bb7-9275-7d26a7bc5cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:01.320858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781e52d-3c01-4bb7-9275-7d26a7bc5cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:01.321170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781e52d-3c01-4bb7-9275-7d26a7bc5cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:01.322542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781e52d-3c01-4bb7-9275-7d26a7bc5cc8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:01.340210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 241/476] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:01.348481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e52d-3c01-4bb7-9275-7d26a7bc5cc8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:01.373192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8445f-87a5-4286-a3ec-5ba0d02656c2 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:01.376373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:01.415907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:01.416894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:01.417402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:01.417825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:01.418809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:01.420380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:01.429074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.sql.core [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 23:36:01.429074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 23:36:01.429074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [parameters: {'id': 'fff6c52af971490ca5115e4b50bdf958', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 23:36:01.429074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86604) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application conn.query(q) Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result.read() Jun 24 23:36:01.448519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 23:36:01.449506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application next(self.gen) Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application yield resource Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application next(self.gen) Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise self.value Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application session.commit() Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 23:36:01.450412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.session.flush() Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application compat.raise_( Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 23:36:01.451243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 23:36:01.452097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application conn.query(q) Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result.read() Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 23:36:01.453149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [parameters: {'id': 'fff6c52af971490ca5115e4b50bdf958', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:01.454658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 23:36:01.455623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:01.458666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 236/477] 10.222.0.47 () {60 vars in 1131 bytes} [Fri Jun 24 23:36:01 2022] POST /identity/v3/roles => generated 144 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:01.464895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:01.468641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781e52d-3c01-4bb7-9275-7d26a7bc5cc8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:01.507223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f79bc-cfe1-43c2-890d-cece35a9cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:01.508074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f79bc-cfe1-43c2-890d-cece35a9cb45 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:01.508479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f79bc-cfe1-43c2-890d-cece35a9cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:01.509010 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f79bc-cfe1-43c2-890d-cece35a9cb45 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:01.509963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f79bc-cfe1-43c2-890d-cece35a9cb45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:01.512415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f79bc-cfe1-43c2-890d-cece35a9cb45 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:01.536042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 242/478] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:36:01 2022] POST /identity/v3/users => generated 249 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:36:01.658509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5493c-05eb-442a-b288-ae031db74731 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:01.694068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:01.721403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf3551-c46e-425d-b60e-b46eabba6f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:01.722270 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf3551-c46e-425d-b60e-b46eabba6f6f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:01.722562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf3551-c46e-425d-b60e-b46eabba6f6f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:01.722846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf3551-c46e-425d-b60e-b46eabba6f6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:01.740302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf3551-c46e-425d-b60e-b46eabba6f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-6bcf3551-c46e-425d-b60e-b46eabba6f6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:01.748716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:01.749647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:01.752191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 237/479] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:01.756818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f79bc-cfe1-43c2-890d-cece35a9cb45 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:01.846014 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:01.871154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f79bc-cfe1-43c2-890d-cece35a9cb45 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:01.909312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d3cd-900c-4c28-b7dd-9cf75ec86b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:01.910062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d3cd-900c-4c28-b7dd-9cf75ec86b87 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:01.910353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d3cd-900c-4c28-b7dd-9cf75ec86b87 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:01.910635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d3cd-900c-4c28-b7dd-9cf75ec86b87 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:01.911415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d3cd-900c-4c28-b7dd-9cf75ec86b87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:01.911730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d3cd-900c-4c28-b7dd-9cf75ec86b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:01.913171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d3cd-900c-4c28-b7dd-9cf75ec86b87 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:01.922676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 243/480] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:03.014476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb1951-b408-4522-adb0-c2f4e6f8fb94 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:03.014980 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb1951-b408-4522-adb0-c2f4e6f8fb94 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:03.015051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb1951-b408-4522-adb0-c2f4e6f8fb94 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:03.016704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 238/481] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:03.022164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c2e86-b323-43e3-badb-12ccdc3092f3 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:03.022560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c2e86-b323-43e3-badb-12ccdc3092f3 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:03.022722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c2e86-b323-43e3-badb-12ccdc3092f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:03.049605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-234c2e86-b323-43e3-badb-12ccdc3092f3 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:03.050818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-234c2e86-b323-43e3-badb-12ccdc3092f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:03.151207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 244/482] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:03.188974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb1951-b408-4522-adb0-c2f4e6f8fb94 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:03.331631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb1951-b408-4522-adb0-c2f4e6f8fb94 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:03.381546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38029dfc-c6a6-489a-9cd7-fd9901b47cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:03.382354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029dfc-c6a6-489a-9cd7-fd9901b47cf0 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:03.382640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029dfc-c6a6-489a-9cd7-fd9901b47cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:03.382917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029dfc-c6a6-489a-9cd7-fd9901b47cf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:03.401675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38029dfc-c6a6-489a-9cd7-fd9901b47cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-38029dfc-c6a6-489a-9cd7-fd9901b47cf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:03.410143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:03.411005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:03.412405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 239/483] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:03.417147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c2e86-b323-43e3-badb-12ccdc3092f3 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:03.516806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:03.556827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c2e86-b323-43e3-badb-12ccdc3092f3 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:03.586319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef227c7a-3e3a-4090-a2e1-bba48ce00775 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:03.586932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef227c7a-3e3a-4090-a2e1-bba48ce00775 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:03.587424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef227c7a-3e3a-4090-a2e1-bba48ce00775 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:03.590268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 240/484] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:03.595091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352995cf-0c13-473a-b440-83c28bb04f22 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:03.595607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352995cf-0c13-473a-b440-83c28bb04f22 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:03.596098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352995cf-0c13-473a-b440-83c28bb04f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:03.611145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941608cf-2d13-4654-b6f0-5e7fda9731d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:03.611821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941608cf-2d13-4654-b6f0-5e7fda9731d9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:03.612108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941608cf-2d13-4654-b6f0-5e7fda9731d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:03.612388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941608cf-2d13-4654-b6f0-5e7fda9731d9 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:03.613182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941608cf-2d13-4654-b6f0-5e7fda9731d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:03.613493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941608cf-2d13-4654-b6f0-5e7fda9731d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:03.614800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-352995cf-0c13-473a-b440-83c28bb04f22 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:03.614907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941608cf-2d13-4654-b6f0-5e7fda9731d9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:03.616695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-352995cf-0c13-473a-b440-83c28bb04f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:03.635554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 245/485] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:03.639441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941608cf-2d13-4654-b6f0-5e7fda9731d9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:03.676105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:03.748147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 241/486] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:36:03.752078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6a28a-f0b6-4e94-b8e3-24b1d5bd2423 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:03.752544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6a28a-f0b6-4e94-b8e3-24b1d5bd2423 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:03.753059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6a28a-f0b6-4e94-b8e3-24b1d5bd2423 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:03.755005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 242/487] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:03.760088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2202ca3-88a9-4349-8851-feb1403dca2a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:03.760620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2202ca3-88a9-4349-8851-feb1403dca2a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:03.761080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2202ca3-88a9-4349-8851-feb1403dca2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:03.778801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d2202ca3-88a9-4349-8851-feb1403dca2a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:03.780409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d2202ca3-88a9-4349-8851-feb1403dca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:03.822253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941608cf-2d13-4654-b6f0-5e7fda9731d9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:03.850685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 243/488] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:36:03.852645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2202ca3-88a9-4349-8851-feb1403dca2a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:03.854390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:03.862409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dec796-7312-4a26-9a2a-ec8f931f7c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:03.863224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dec796-7312-4a26-9a2a-ec8f931f7c6e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:03.863504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dec796-7312-4a26-9a2a-ec8f931f7c6e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:03.863831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dec796-7312-4a26-9a2a-ec8f931f7c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:03.886142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dec796-7312-4a26-9a2a-ec8f931f7c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:03.887641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dec796-7312-4a26-9a2a-ec8f931f7c6e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-26dec796-7312-4a26-9a2a-ec8f931f7c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:03.915510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:03.916394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:03.917241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:03.917241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:03.917241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:03.917241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:03.917241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 246/489] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:03.920622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dec796-7312-4a26-9a2a-ec8f931f7c6e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:03.963968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:03.967833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2202ca3-88a9-4349-8851-feb1403dca2a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.004717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a4ea9-c83e-4166-9c41-8b85476ba8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.005827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a4ea9-c83e-4166-9c41-8b85476ba8e0 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.006289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a4ea9-c83e-4166-9c41-8b85476ba8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.006745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a4ea9-c83e-4166-9c41-8b85476ba8e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.019047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a4ea9-c83e-4166-9c41-8b85476ba8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-671a4ea9-c83e-4166-9c41-8b85476ba8e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:04.028015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:04.028933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:04.029701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:04.033644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dec796-7312-4a26-9a2a-ec8f931f7c6e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.034729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 244/490] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:04.036566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a4ea9-c83e-4166-9c41-8b85476ba8e0 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.079022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a77cf-f2a9-4870-9bdb-ed593131f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.079843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a77cf-f2a9-4870-9bdb-ed593131f5f0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.080172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a77cf-f2a9-4870-9bdb-ed593131f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.080449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a77cf-f2a9-4870-9bdb-ed593131f5f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.091796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a77cf-f2a9-4870-9bdb-ed593131f5f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-5b9a77cf-f2a9-4870-9bdb-ed593131f5f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:04.100371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:04.101374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:04.102156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 247/491] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:04.107040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a77cf-f2a9-4870-9bdb-ed593131f5f0 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.151916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a4ea9-c83e-4166-9c41-8b85476ba8e0 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.155567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:04.190201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9970b-6adf-4610-930d-ab89d6a50522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.191240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9970b-6adf-4610-930d-ab89d6a50522 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.191723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9970b-6adf-4610-930d-ab89d6a50522 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.192236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9970b-6adf-4610-930d-ab89d6a50522 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.193246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9970b-6adf-4610-930d-ab89d6a50522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:04.193753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9970b-6adf-4610-930d-ab89d6a50522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:04.195365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9970b-6adf-4610-930d-ab89d6a50522 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:04.209106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 245/492] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:04.214572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9970b-6adf-4610-930d-ab89d6a50522 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.216515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a77cf-f2a9-4870-9bdb-ed593131f5f0 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.235613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:04.255213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28708c31-0c92-439e-a69d-7f5a82049c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.255783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28708c31-0c92-439e-a69d-7f5a82049c3a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.256064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28708c31-0c92-439e-a69d-7f5a82049c3a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.256340 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28708c31-0c92-439e-a69d-7f5a82049c3a None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.257129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28708c31-0c92-439e-a69d-7f5a82049c3a None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:04.257437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28708c31-0c92-439e-a69d-7f5a82049c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:04.258805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28708c31-0c92-439e-a69d-7f5a82049c3a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:04.268498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 248/493] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:04.273723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28708c31-0c92-439e-a69d-7f5a82049c3a None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.326991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9970b-6adf-4610-930d-ab89d6a50522 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.362912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cec1c7-349f-4a36-b712-a2a31ba9a804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.363508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cec1c7-349f-4a36-b712-a2a31ba9a804 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.363795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cec1c7-349f-4a36-b712-a2a31ba9a804 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.364076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cec1c7-349f-4a36-b712-a2a31ba9a804 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.364853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cec1c7-349f-4a36-b712-a2a31ba9a804 None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:04.365165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cec1c7-349f-4a36-b712-a2a31ba9a804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:04.366560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cec1c7-349f-4a36-b712-a2a31ba9a804 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:04.376618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 246/494] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:04.376618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cec1c7-349f-4a36-b712-a2a31ba9a804 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.387276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28708c31-0c92-439e-a69d-7f5a82049c3a None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.394976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:04.423348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8722-4732-4870-83d6-53a63b104299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.424144 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8722-4732-4870-83d6-53a63b104299 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.424423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8722-4732-4870-83d6-53a63b104299 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.424729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8722-4732-4870-83d6-53a63b104299 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.433263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8722-4732-4870-83d6-53a63b104299 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:04.434751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b8722-4732-4870-83d6-53a63b104299 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-779b8722-4732-4870-83d6-53a63b104299 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:04.451274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:04.452155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:04.453085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 249/495] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:04.454135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49887389-cd2b-4591-8bf1-03e5046fdf5c None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.454229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49887389-cd2b-4591-8bf1-03e5046fdf5c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.454491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49887389-cd2b-4591-8bf1-03e5046fdf5c None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.459764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 250/496] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:04.459764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49887389-cd2b-4591-8bf1-03e5046fdf5c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.472066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:04.493324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cec1c7-349f-4a36-b712-a2a31ba9a804 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.528146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4614fc-c6a7-4ef1-a2bc-c9344e243933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.529008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4614fc-c6a7-4ef1-a2bc-c9344e243933 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.529304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4614fc-c6a7-4ef1-a2bc-c9344e243933 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.529583 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4614fc-c6a7-4ef1-a2bc-c9344e243933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.540441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4614fc-c6a7-4ef1-a2bc-c9344e243933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:04.541940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4614fc-c6a7-4ef1-a2bc-c9344e243933 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-fe4614fc-c6a7-4ef1-a2bc-c9344e243933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:04.559846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:04.560767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:04.561588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:04.561588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:04.561588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:04.561588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:04.561588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 247/497] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:04.561869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4614fc-c6a7-4ef1-a2bc-c9344e243933 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.565663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49887389-cd2b-4591-8bf1-03e5046fdf5c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.599260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6de739-0c07-4910-9aa6-145e3a23bf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.600037 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6de739-0c07-4910-9aa6-145e3a23bf4f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.600317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6de739-0c07-4910-9aa6-145e3a23bf4f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.600609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6de739-0c07-4910-9aa6-145e3a23bf4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.611134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6de739-0c07-4910-9aa6-145e3a23bf4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:04.612612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6de739-0c07-4910-9aa6-145e3a23bf4f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ec6de739-0c07-4910-9aa6-145e3a23bf4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:04.629969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:04.630969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:04.632005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:04.632005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:04.632005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:04.632005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:04.632005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 251/498] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:04.632316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6f348-bf51-45de-859d-ab83a27e5e52 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.632565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6f348-bf51-45de-859d-ab83a27e5e52 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.632868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6f348-bf51-45de-859d-ab83a27e5e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.649699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d7b6f348-bf51-45de-859d-ab83a27e5e52 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:04.650874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d7b6f348-bf51-45de-859d-ab83a27e5e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:04.663968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:04.665946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4614fc-c6a7-4ef1-a2bc-c9344e243933 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.695185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:04.700559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f7937-aab8-42b8-857b-53b189e36a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.701250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f7937-aab8-42b8-857b-53b189e36a46 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.701534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f7937-aab8-42b8-857b-53b189e36a46 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.701811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f7937-aab8-42b8-857b-53b189e36a46 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.702646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f7937-aab8-42b8-857b-53b189e36a46 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:04.702954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f7937-aab8-42b8-857b-53b189e36a46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:04.704631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f7937-aab8-42b8-857b-53b189e36a46 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:04.717887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 252/499] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:04.719534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfa678-81fc-4674-afee-176f1f0a98d2 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.719922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfa678-81fc-4674-afee-176f1f0a98d2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.720206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfa678-81fc-4674-afee-176f1f0a98d2 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.721790 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 253/500] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:04.722487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfa678-81fc-4674-afee-176f1f0a98d2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.749489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 248/501] 10.222.0.47 () {56 vars in 1137 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:04.749489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f7937-aab8-42b8-857b-53b189e36a46 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.838661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfa678-81fc-4674-afee-176f1f0a98d2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.848316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f7937-aab8-42b8-857b-53b189e36a46 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:04.871253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc0a9d-d3cb-4490-b9ed-950f586b9b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.871912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc0a9d-d3cb-4490-b9ed-950f586b9b2e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.872189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc0a9d-d3cb-4490-b9ed-950f586b9b2e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.872628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc0a9d-d3cb-4490-b9ed-950f586b9b2e None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.873205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc0a9d-d3cb-4490-b9ed-950f586b9b2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:04.873504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc0a9d-d3cb-4490-b9ed-950f586b9b2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:04.875373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc0a9d-d3cb-4490-b9ed-950f586b9b2e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:04.883655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144aabc-05bb-4ef6-b8e6-29d4d23183c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:04.884256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144aabc-05bb-4ef6-b8e6-29d4d23183c9 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.884541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144aabc-05bb-4ef6-b8e6-29d4d23183c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.884854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144aabc-05bb-4ef6-b8e6-29d4d23183c9 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.885571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144aabc-05bb-4ef6-b8e6-29d4d23183c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:04.885881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144aabc-05bb-4ef6-b8e6-29d4d23183c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:04.886309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 254/502] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:04.887264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144aabc-05bb-4ef6-b8e6-29d4d23183c9 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:04.887559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfca73-f592-4a59-a30a-9ca8c9d5cada None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:04.887850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfca73-f592-4a59-a30a-9ca8c9d5cada None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:04.888128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfca73-f592-4a59-a30a-9ca8c9d5cada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:04.900087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 249/503] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:04.900947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144aabc-05bb-4ef6-b8e6-29d4d23183c9 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:04.905323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8adfca73-f592-4a59-a30a-9ca8c9d5cada None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:04.906524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8adfca73-f592-4a59-a30a-9ca8c9d5cada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:04.908512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:04.954465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:04.978690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 255/504] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:04.979531 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfca73-f592-4a59-a30a-9ca8c9d5cada None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.008389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144aabc-05bb-4ef6-b8e6-29d4d23183c9 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.042040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa4c3d-c94b-40c1-a5f2-40e2a5ace945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.042832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4c3d-c94b-40c1-a5f2-40e2a5ace945 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.043116 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4c3d-c94b-40c1-a5f2-40e2a5ace945 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.043393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4c3d-c94b-40c1-a5f2-40e2a5ace945 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.053404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa4c3d-c94b-40c1-a5f2-40e2a5ace945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-37aa4c3d-c94b-40c1-a5f2-40e2a5ace945 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:05.061015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:05.062002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 250/505] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:05.063792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa4c3d-c94b-40c1-a5f2-40e2a5ace945 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.092188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfca73-f592-4a59-a30a-9ca8c9d5cada None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.125356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be55f04-f447-4feb-9f0b-374cded62f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.126104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be55f04-f447-4feb-9f0b-374cded62f0e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.126396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be55f04-f447-4feb-9f0b-374cded62f0e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.126709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be55f04-f447-4feb-9f0b-374cded62f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.134776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:05.162525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa4c3d-c94b-40c1-a5f2-40e2a5ace945 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.174244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:05.195068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be55f04-f447-4feb-9f0b-374cded62f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.196531 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be55f04-f447-4feb-9f0b-374cded62f0e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.196760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c57e-976b-4a97-9286-553a55eeeff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.197551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c57e-976b-4a97-9286-553a55eeeff0 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.197838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c57e-976b-4a97-9286-553a55eeeff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.198117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c57e-976b-4a97-9286-553a55eeeff0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.206333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58c57e-976b-4a97-9286-553a55eeeff0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.207781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58c57e-976b-4a97-9286-553a55eeeff0 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-6f58c57e-976b-4a97-9286-553a55eeeff0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:05.222853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:05.223826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:05.224769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 251/506] 10.222.0.47 () {56 vars in 1093 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:05.225320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c57e-976b-4a97-9286-553a55eeeff0 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.299771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 256/507] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:05.300630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be55f04-f447-4feb-9f0b-374cded62f0e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.328850 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c57e-976b-4a97-9286-553a55eeeff0 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.341699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:05.365973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963187f9-8c71-4ee9-9565-66a56b82cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.367144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963187f9-8c71-4ee9-9565-66a56b82cae7 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.367432 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963187f9-8c71-4ee9-9565-66a56b82cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.367719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963187f9-8c71-4ee9-9565-66a56b82cae7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.376033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963187f9-8c71-4ee9-9565-66a56b82cae7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.377517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963187f9-8c71-4ee9-9565-66a56b82cae7 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-963187f9-8c71-4ee9-9565-66a56b82cae7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:05.391522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:05.392380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 252/508] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:05.393463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963187f9-8c71-4ee9-9565-66a56b82cae7 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.406960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be55f04-f447-4feb-9f0b-374cded62f0e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.423503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:05.440951 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd7cc2-58cf-4acc-bb00-965b5fc373f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.441751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd7cc2-58cf-4acc-bb00-965b5fc373f4 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.442035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd7cc2-58cf-4acc-bb00-965b5fc373f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.442315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd7cc2-58cf-4acc-bb00-965b5fc373f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.452061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd7cc2-58cf-4acc-bb00-965b5fc373f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.453833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd7cc2-58cf-4acc-bb00-965b5fc373f4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-7ebd7cc2-58cf-4acc-bb00-965b5fc373f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:05.470954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:05.471862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:05.472727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:05.472727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:05.472727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:05.472727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:05.472727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 257/509] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:05.472975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd7cc2-58cf-4acc-bb00-965b5fc373f4 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.498012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963187f9-8c71-4ee9-9565-66a56b82cae7 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.533210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed7073-428b-408a-8ae2-e74cbe8c071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.533874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed7073-428b-408a-8ae2-e74cbe8c071a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.534166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed7073-428b-408a-8ae2-e74cbe8c071a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.534444 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed7073-428b-408a-8ae2-e74cbe8c071a None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.535190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed7073-428b-408a-8ae2-e74cbe8c071a None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:05.535502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed7073-428b-408a-8ae2-e74cbe8c071a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.536915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eed7073-428b-408a-8ae2-e74cbe8c071a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.544569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 253/510] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:05.545384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed7073-428b-408a-8ae2-e74cbe8c071a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.561719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:05.572029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd7cc2-58cf-4acc-bb00-965b5fc373f4 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.604704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043bd57-948e-46af-8810-3d2dc8aef052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.605371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043bd57-948e-46af-8810-3d2dc8aef052 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.605655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043bd57-948e-46af-8810-3d2dc8aef052 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.605924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043bd57-948e-46af-8810-3d2dc8aef052 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.606761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043bd57-948e-46af-8810-3d2dc8aef052 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:05.607069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043bd57-948e-46af-8810-3d2dc8aef052 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.608497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043bd57-948e-46af-8810-3d2dc8aef052 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.637024 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:05.646056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 258/511] 10.222.0.47 () {56 vars in 1133 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:05.646643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed7073-428b-408a-8ae2-e74cbe8c071a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.646825 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043bd57-948e-46af-8810-3d2dc8aef052 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.682292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e058f4f-75ed-4734-b802-86cc991e1810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.683077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e058f4f-75ed-4734-b802-86cc991e1810 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.683373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e058f4f-75ed-4734-b802-86cc991e1810 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.683642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e058f4f-75ed-4734-b802-86cc991e1810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.694462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e058f4f-75ed-4734-b802-86cc991e1810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.695941 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e058f4f-75ed-4734-b802-86cc991e1810 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-7e058f4f-75ed-4734-b802-86cc991e1810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:05.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:05.713573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:05.714410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:05.714410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:05.714410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:05.714410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:05.714410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 254/512] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:05.714679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e058f4f-75ed-4734-b802-86cc991e1810 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.749726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043bd57-948e-46af-8810-3d2dc8aef052 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.783294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1612fff-2c61-4a1b-a85b-da39f672bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.784712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1612fff-2c61-4a1b-a85b-da39f672bde6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.785004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1612fff-2c61-4a1b-a85b-da39f672bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.785287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1612fff-2c61-4a1b-a85b-da39f672bde6 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.786139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1612fff-2c61-4a1b-a85b-da39f672bde6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:05.786455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1612fff-2c61-4a1b-a85b-da39f672bde6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.787889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1612fff-2c61-4a1b-a85b-da39f672bde6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.806246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 259/513] 10.222.0.47 () {56 vars in 1137 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:05.807047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1612fff-2c61-4a1b-a85b-da39f672bde6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.815101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:05.818319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e058f4f-75ed-4734-b802-86cc991e1810 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.852257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bc4f7-2018-49dc-84a8-0ff1bd0c910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.852955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc4f7-2018-49dc-84a8-0ff1bd0c910f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.853236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc4f7-2018-49dc-84a8-0ff1bd0c910f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.853512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc4f7-2018-49dc-84a8-0ff1bd0c910f None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.854226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bc4f7-2018-49dc-84a8-0ff1bd0c910f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:05.854533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bc4f7-2018-49dc-84a8-0ff1bd0c910f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.855899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bc4f7-2018-49dc-84a8-0ff1bd0c910f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.867974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 255/514] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:05.868790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bc4f7-2018-49dc-84a8-0ff1bd0c910f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:05.872945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:05.910794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1612fff-2c61-4a1b-a85b-da39f672bde6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.945200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a6534-55bc-4616-bf00-405686e92ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:05.945988 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a6534-55bc-4616-bf00-405686e92ae2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:05.946259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a6534-55bc-4616-bf00-405686e92ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:05.946529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a6534-55bc-4616-bf00-405686e92ae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:05.959060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a6534-55bc-4616-bf00-405686e92ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:05.960524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a6534-55bc-4616-bf00-405686e92ae2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:05.967947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bc4f7-2018-49dc-84a8-0ff1bd0c910f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-798a6534-55bc-4616-bf00-405686e92ae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:05.977722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:05.978602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:05.979571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:05.979571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:05.979571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:05.979571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:05.979571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 260/515] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:05.979995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a6534-55bc-4616-bf00-405686e92ae2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.005452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffadc3-783e-4b8e-86e8-979f19cadbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:06.006108 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffadc3-783e-4b8e-86e8-979f19cadbdb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:06.006393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffadc3-783e-4b8e-86e8-979f19cadbdb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:06.006677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffadc3-783e-4b8e-86e8-979f19cadbdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:06.038694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:06.076129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffadc3-783e-4b8e-86e8-979f19cadbdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:06.077623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffadc3-783e-4b8e-86e8-979f19cadbdb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:06.080742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:06.081104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a6534-55bc-4616-bf00-405686e92ae2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:06.114893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91fb5f-99c8-41e0-8657-3fa6800ba8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:06.115551 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91fb5f-99c8-41e0-8657-3fa6800ba8f2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:06.115831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91fb5f-99c8-41e0-8657-3fa6800ba8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:06.116103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91fb5f-99c8-41e0-8657-3fa6800ba8f2 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:06.116846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91fb5f-99c8-41e0-8657-3fa6800ba8f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:06.117153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91fb5f-99c8-41e0-8657-3fa6800ba8f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:06.118504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91fb5f-99c8-41e0-8657-3fa6800ba8f2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:06.130541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 261/516] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:06.131196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91fb5f-99c8-41e0-8657-3fa6800ba8f2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.177271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 256/517] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:06.178008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffadc3-783e-4b8e-86e8-979f19cadbdb None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.237601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91fb5f-99c8-41e0-8657-3fa6800ba8f2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:06.254216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:06.272050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555d122-47f9-4709-8906-06e0e7e0da17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:06.272732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555d122-47f9-4709-8906-06e0e7e0da17 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:06.273274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555d122-47f9-4709-8906-06e0e7e0da17 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:06.273555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555d122-47f9-4709-8906-06e0e7e0da17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:06.277484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffadc3-783e-4b8e-86e8-979f19cadbdb None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:06.280463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:06.311846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9489788-7b06-4c6f-9737-8d9b2d5de43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:06.313048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9489788-7b06-4c6f-9737-8d9b2d5de43e None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:06.313322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9489788-7b06-4c6f-9737-8d9b2d5de43e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:06.313595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9489788-7b06-4c6f-9737-8d9b2d5de43e None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:06.314294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9489788-7b06-4c6f-9737-8d9b2d5de43e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:06.315724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9489788-7b06-4c6f-9737-8d9b2d5de43e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:06.335573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 257/518] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:36:06 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:06.336368 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9489788-7b06-4c6f-9737-8d9b2d5de43e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.359650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555d122-47f9-4709-8906-06e0e7e0da17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:06.361368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555d122-47f9-4709-8906-06e0e7e0da17 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:06.454536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9489788-7b06-4c6f-9737-8d9b2d5de43e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:06.487873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 262/519] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:06.487873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555d122-47f9-4709-8906-06e0e7e0da17 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.503444 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173059ad-b6ea-4b50-8e7d-08bb198397fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:06.504392 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173059ad-b6ea-4b50-8e7d-08bb198397fb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:06.504524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173059ad-b6ea-4b50-8e7d-08bb198397fb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:06.504837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173059ad-b6ea-4b50-8e7d-08bb198397fb None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:06.505568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173059ad-b6ea-4b50-8e7d-08bb198397fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:06.505878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173059ad-b6ea-4b50-8e7d-08bb198397fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:06.507246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173059ad-b6ea-4b50-8e7d-08bb198397fb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:06.508409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:06.521091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 258/520] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:06.522152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173059ad-b6ea-4b50-8e7d-08bb198397fb None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.550182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:06.604463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555d122-47f9-4709-8906-06e0e7e0da17 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:06.637053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173059ad-b6ea-4b50-8e7d-08bb198397fb None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:06.640095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79b033-bec1-4b90-a407-317e3669458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:06.641496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79b033-bec1-4b90-a407-317e3669458c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:06.641812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79b033-bec1-4b90-a407-317e3669458c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:06.642039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79b033-bec1-4b90-a407-317e3669458c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:06.648974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79b033-bec1-4b90-a407-317e3669458c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:06.650405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79b033-bec1-4b90-a407-317e3669458c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-0b79b033-bec1-4b90-a407-317e3669458c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:06.664484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:06.665351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:06.666164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:06.666164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:06.666164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:06.666164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:06.666164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:06.666164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:06.666164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:06.666164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:06.666164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 263/521] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:06.666616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79b033-bec1-4b90-a407-317e3669458c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.680234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2247e3-55c3-48ed-b5b8-4254933df1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:06.681125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2247e3-55c3-48ed-b5b8-4254933df1fc None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:06.681411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2247e3-55c3-48ed-b5b8-4254933df1fc None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:06.681689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2247e3-55c3-48ed-b5b8-4254933df1fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:06.694729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2247e3-55c3-48ed-b5b8-4254933df1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:06.696556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2247e3-55c3-48ed-b5b8-4254933df1fc None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-9f2247e3-55c3-48ed-b5b8-4254933df1fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:06.716727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:06.717598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:06.723593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:06.723593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:06.723593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:06.723593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:06.723593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 259/522] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:06.723593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2247e3-55c3-48ed-b5b8-4254933df1fc None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.756843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:06.783035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79b033-bec1-4b90-a407-317e3669458c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:06.799807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:06.819649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562fa4d-44cb-4d58-b69b-10554ba889a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:06.820447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562fa4d-44cb-4d58-b69b-10554ba889a5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:06.820772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562fa4d-44cb-4d58-b69b-10554ba889a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:06.821050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562fa4d-44cb-4d58-b69b-10554ba889a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:06.831727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562fa4d-44cb-4d58-b69b-10554ba889a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:06.833210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1562fa4d-44cb-4d58-b69b-10554ba889a5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:06.834071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2247e3-55c3-48ed-b5b8-4254933df1fc None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-1562fa4d-44cb-4d58-b69b-10554ba889a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:06.850657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:06.851508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:06.852329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:06.852329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:06.852329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:06.852329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:06.852329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 264/523] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:06.853139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562fa4d-44cb-4d58-b69b-10554ba889a5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.877240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4046d96-4daf-440a-b715-ce4b135dc9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:06.878034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4046d96-4daf-440a-b715-ce4b135dc9cb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:06.878316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4046d96-4daf-440a-b715-ce4b135dc9cb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:06.878591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4046d96-4daf-440a-b715-ce4b135dc9cb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:06.886500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4046d96-4daf-440a-b715-ce4b135dc9cb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:06.887953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4046d96-4daf-440a-b715-ce4b135dc9cb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-a4046d96-4daf-440a-b715-ce4b135dc9cb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:06.904429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:06.905527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:06.906724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 260/524] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:06.907255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4046d96-4daf-440a-b715-ce4b135dc9cb None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:06.972802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1562fa4d-44cb-4d58-b69b-10554ba889a5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:07.002004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:07.008055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a46e2-95e7-4b59-8a16-56d5baeac7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:07.009136 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a46e2-95e7-4b59-8a16-56d5baeac7b6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:07.009296 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a46e2-95e7-4b59-8a16-56d5baeac7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:07.009550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a46e2-95e7-4b59-8a16-56d5baeac7b6 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:07.010364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a46e2-95e7-4b59-8a16-56d5baeac7b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:07.011245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a46e2-95e7-4b59-8a16-56d5baeac7b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:07.013294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a46e2-95e7-4b59-8a16-56d5baeac7b6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:07.040603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 265/525] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:07.040792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4046d96-4daf-440a-b715-ce4b135dc9cb None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:07.040883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a46e2-95e7-4b59-8a16-56d5baeac7b6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:07.072201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:07.078994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01efef-9b17-48e9-8d97-34b5fc46709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:07.079686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01efef-9b17-48e9-8d97-34b5fc46709d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:07.079954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01efef-9b17-48e9-8d97-34b5fc46709d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:07.080236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01efef-9b17-48e9-8d97-34b5fc46709d None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:07.081512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01efef-9b17-48e9-8d97-34b5fc46709d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:07.081512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01efef-9b17-48e9-8d97-34b5fc46709d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:07.082670 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01efef-9b17-48e9-8d97-34b5fc46709d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:07.094812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 261/526] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:07.095645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01efef-9b17-48e9-8d97-34b5fc46709d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:07.155173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a46e2-95e7-4b59-8a16-56d5baeac7b6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:07.191800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6fa8c-cf65-4a4a-975a-2092284e7ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:07.192470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6fa8c-cf65-4a4a-975a-2092284e7ec9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:07.192789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6fa8c-cf65-4a4a-975a-2092284e7ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:07.193075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6fa8c-cf65-4a4a-975a-2092284e7ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:07.194222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6fa8c-cf65-4a4a-975a-2092284e7ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:07.194539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6fa8c-cf65-4a4a-975a-2092284e7ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:07.196079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6fa8c-cf65-4a4a-975a-2092284e7ec9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:07.206126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01efef-9b17-48e9-8d97-34b5fc46709d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:07.209041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 266/527] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:07.209372 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6fa8c-cf65-4a4a-975a-2092284e7ec9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:07.245071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16610e-620b-47d3-a671-06a0d58300c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:07.245662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16610e-620b-47d3-a671-06a0d58300c2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:07.245944 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16610e-620b-47d3-a671-06a0d58300c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:07.246218 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16610e-620b-47d3-a671-06a0d58300c2 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:07.247057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16610e-620b-47d3-a671-06a0d58300c2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:07.247360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16610e-620b-47d3-a671-06a0d58300c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:07.248807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16610e-620b-47d3-a671-06a0d58300c2 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:07.262975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:07.271760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 262/528] 10.222.0.47 () {56 vars in 1139 bytes} [Fri Jun 24 23:36:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:07.271760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16610e-620b-47d3-a671-06a0d58300c2 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:07.325897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6fa8c-cf65-4a4a-975a-2092284e7ec9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:07.331686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:07.360947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c238e9-50f0-4e81-a15d-64eefbc576eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:07.361672 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c238e9-50f0-4e81-a15d-64eefbc576eb None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:07.361953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c238e9-50f0-4e81-a15d-64eefbc576eb None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:07.362283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c238e9-50f0-4e81-a15d-64eefbc576eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:07.363124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c238e9-50f0-4e81-a15d-64eefbc576eb None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=71e5ef9dc32f494791ba4addb5151232) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:07.363438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c238e9-50f0-4e81-a15d-64eefbc576eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:07.364904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c238e9-50f0-4e81-a15d-64eefbc576eb None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:07.380624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 267/529] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:07 2022] GET /identity/v3/role_assignments?user.id=71e5ef9dc32f494791ba4addb5151232&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:07.380624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c238e9-50f0-4e81-a15d-64eefbc576eb None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:07.403142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16610e-620b-47d3-a671-06a0d58300c2 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:07.463094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15767e2-8bad-40f9-bd62-c2091fee9bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:07.464231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15767e2-8bad-40f9-bd62-c2091fee9bba None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:07.464730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15767e2-8bad-40f9-bd62-c2091fee9bba None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:07.465199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15767e2-8bad-40f9-bd62-c2091fee9bba None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:07.478249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15767e2-8bad-40f9-bd62-c2091fee9bba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:07.480822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15767e2-8bad-40f9-bd62-c2091fee9bba None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-d15767e2-8bad-40f9-bd62-c2091fee9bba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:07.509314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:07.510202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:07.511008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:07.511008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:07.511008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:07.511008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:07.511008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:07.511008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:07.511008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:07.511008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:07.517177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 263/530] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:07.518769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15767e2-8bad-40f9-bd62-c2091fee9bba None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:07.568351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c238e9-50f0-4e81-a15d-64eefbc576eb None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:07.592896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:07.621584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0029c-986c-40b3-a207-00dc2f490d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:07.622694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0029c-986c-40b3-a207-00dc2f490d60 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:07.622979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0029c-986c-40b3-a207-00dc2f490d60 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:07.623295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0029c-986c-40b3-a207-00dc2f490d60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:07.632390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0029c-986c-40b3-a207-00dc2f490d60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:07.634291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0029c-986c-40b3-a207-00dc2f490d60 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-96a0029c-986c-40b3-a207-00dc2f490d60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:07.654416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:07.655279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:07.656786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:07.656786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:07.656786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:07.656786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:07.656786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:07.656786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:07.656786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:07.656786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:07.657631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 268/531] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:07.660604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0029c-986c-40b3-a207-00dc2f490d60 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:07.664739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15767e2-8bad-40f9-bd62-c2091fee9bba None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:07.678777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:07.704769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403246d-8c3d-4dd2-9ff2-398bed98f85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:07.705434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403246d-8c3d-4dd2-9ff2-398bed98f85e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:07.705725 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403246d-8c3d-4dd2-9ff2-398bed98f85e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:07.706009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403246d-8c3d-4dd2-9ff2-398bed98f85e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:07.774531 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0029c-986c-40b3-a207-00dc2f490d60 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:07.780889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403246d-8c3d-4dd2-9ff2-398bed98f85e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:07.783042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403246d-8c3d-4dd2-9ff2-398bed98f85e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:07.813001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f79877-d0b8-4b54-b946-d27ab0cca452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:07.813696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f79877-d0b8-4b54-b946-d27ab0cca452 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:07.813978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f79877-d0b8-4b54-b946-d27ab0cca452 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:07.814255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f79877-d0b8-4b54-b946-d27ab0cca452 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:07.815244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f79877-d0b8-4b54-b946-d27ab0cca452 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:07.815559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f79877-d0b8-4b54-b946-d27ab0cca452 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:07.817004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f79877-d0b8-4b54-b946-d27ab0cca452 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:07.829921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 269/532] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:07.830761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f79877-d0b8-4b54-b946-d27ab0cca452 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:07.866959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:07.902163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 264/533] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:07.903428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403246d-8c3d-4dd2-9ff2-398bed98f85e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:07.918478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:07.951541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f79877-d0b8-4b54-b946-d27ab0cca452 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:08.000260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c653784-4bfe-4942-b7e2-8119c11b40c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:08.002416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c653784-4bfe-4942-b7e2-8119c11b40c3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:08.002891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c653784-4bfe-4942-b7e2-8119c11b40c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:08.003349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c653784-4bfe-4942-b7e2-8119c11b40c3 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:08.004328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c653784-4bfe-4942-b7e2-8119c11b40c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:08.004844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c653784-4bfe-4942-b7e2-8119c11b40c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:08.006967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c653784-4bfe-4942-b7e2-8119c11b40c3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:08.028749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 270/534] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:08.028749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c653784-4bfe-4942-b7e2-8119c11b40c3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:08.046086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403246d-8c3d-4dd2-9ff2-398bed98f85e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:08.096668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851f15-fcdf-4e11-9aa2-135fcfb9f6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:08.098035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851f15-fcdf-4e11-9aa2-135fcfb9f6cf None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:08.098705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851f15-fcdf-4e11-9aa2-135fcfb9f6cf None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:08.099131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851f15-fcdf-4e11-9aa2-135fcfb9f6cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:08.100736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851f15-fcdf-4e11-9aa2-135fcfb9f6cf None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=feb9bec9f1894e179e24a14b81b3f23d) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:08.101205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851f15-fcdf-4e11-9aa2-135fcfb9f6cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:08.103340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851f15-fcdf-4e11-9aa2-135fcfb9f6cf None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:08.110895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 265/535] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:07 2022] GET /identity/v3/role_assignments?user.id=feb9bec9f1894e179e24a14b81b3f23d&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:08.112392 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851f15-fcdf-4e11-9aa2-135fcfb9f6cf None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:08.132251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:08.156565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c653784-4bfe-4942-b7e2-8119c11b40c3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:08.213172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:08.246374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851f15-fcdf-4e11-9aa2-135fcfb9f6cf None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:08.254385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d285e-b826-4fb6-804e-c71c45edeeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:08.255349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d285e-b826-4fb6-804e-c71c45edeeed None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:08.255786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d285e-b826-4fb6-804e-c71c45edeeed None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:08.256161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d285e-b826-4fb6-804e-c71c45edeeed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:08.280288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d285e-b826-4fb6-804e-c71c45edeeed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:08.282409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d285e-b826-4fb6-804e-c71c45edeeed None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:08.290910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed7c4-2b7f-4a46-9a10-c942b01bb9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:08.291790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed7c4-2b7f-4a46-9a10-c942b01bb9a4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:08.292246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed7c4-2b7f-4a46-9a10-c942b01bb9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:08.292727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ed7c4-2b7f-4a46-9a10-c942b01bb9a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:08.293701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ed7c4-2b7f-4a46-9a10-c942b01bb9a4 None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=fb004ddd208849f6a5316c19e9a916aa) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:08.294699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ed7c4-2b7f-4a46-9a10-c942b01bb9a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:08.296330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ed7c4-2b7f-4a46-9a10-c942b01bb9a4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:08.308696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 266/536] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:08 2022] GET /identity/v3/role_assignments?user.id=fb004ddd208849f6a5316c19e9a916aa&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-c85d285e-b826-4fb6-804e-c71c45edeeed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:08.311881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:08.313290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:08.314128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:08.314128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:08.314128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:08.314128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:08.321164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 271/537] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:08.326521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed7c4-2b7f-4a46-9a10-c942b01bb9a4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:08.458379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ed7c4-2b7f-4a46-9a10-c942b01bb9a4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:08.511912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fff22-7426-4031-a6df-f84e65411567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:08.512618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fff22-7426-4031-a6df-f84e65411567 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:08.512901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fff22-7426-4031-a6df-f84e65411567 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:08.513181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fff22-7426-4031-a6df-f84e65411567 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:08.513903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fff22-7426-4031-a6df-f84e65411567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:08.514235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fff22-7426-4031-a6df-f84e65411567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:08.516374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fff22-7426-4031-a6df-f84e65411567 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:08.525541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:08.529171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 267/538] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:08.530179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5982b8-7a2c-4496-968f-1cba81ec540b None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:08.530515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5982b8-7a2c-4496-968f-1cba81ec540b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:08.530741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5982b8-7a2c-4496-968f-1cba81ec540b None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:08.532343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 272/539] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:08.537250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fff22-7426-4031-a6df-f84e65411567 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:08.537747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeab151-034c-4f18-b3f6-7d1486ba4d7d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:08.538032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeab151-034c-4f18-b3f6-7d1486ba4d7d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:08.538312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeab151-034c-4f18-b3f6-7d1486ba4d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:08.561584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-3aeab151-034c-4f18-b3f6-7d1486ba4d7d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:08.563298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3aeab151-034c-4f18-b3f6-7d1486ba4d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:08.677602 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fff22-7426-4031-a6df-f84e65411567 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:08.720711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517d70d-203c-44fa-bb4e-cec5e95e6ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:08.721881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517d70d-203c-44fa-bb4e-cec5e95e6ce9 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:08.723939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517d70d-203c-44fa-bb4e-cec5e95e6ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:08.724464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517d70d-203c-44fa-bb4e-cec5e95e6ce9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:08.737504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517d70d-203c-44fa-bb4e-cec5e95e6ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:08.739057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 273/540] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:08.740918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517d70d-203c-44fa-bb4e-cec5e95e6ce9 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:08.765705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeab151-034c-4f18-b3f6-7d1486ba4d7d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-2517d70d-203c-44fa-bb4e-cec5e95e6ce9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:08.767396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:08.768214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:08.769478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:08.769478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:08.769478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:08.769478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:08.769478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:08.769478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:08.769478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:08.769478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:08.769478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 268/541] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:08.775458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517d70d-203c-44fa-bb4e-cec5e95e6ce9 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:08.843351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:08.892434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517d70d-203c-44fa-bb4e-cec5e95e6ce9 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:08.944613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbabd7-f8b5-4a16-9b6c-db1893b4fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:08.944613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbabd7-f8b5-4a16-9b6c-db1893b4fb73 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:08.944613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbabd7-f8b5-4a16-9b6c-db1893b4fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:08.944613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbabd7-f8b5-4a16-9b6c-db1893b4fb73 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:08.945008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbabd7-f8b5-4a16-9b6c-db1893b4fb73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:08.945008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbabd7-f8b5-4a16-9b6c-db1893b4fb73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:08.945008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbabd7-f8b5-4a16-9b6c-db1893b4fb73 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:08.972988 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:08.989013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 269/542] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:08.991284 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbabd7-f8b5-4a16-9b6c-db1893b4fb73 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:09.044181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeab151-034c-4f18-b3f6-7d1486ba4d7d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:09.120518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573d80-1756-4618-bc7a-6c2fb61353b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:09.121330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573d80-1756-4618-bc7a-6c2fb61353b1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:09.121615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573d80-1756-4618-bc7a-6c2fb61353b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:09.121892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573d80-1756-4618-bc7a-6c2fb61353b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:09.147206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff573d80-1756-4618-bc7a-6c2fb61353b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ff573d80-1756-4618-bc7a-6c2fb61353b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:09.162175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:09.163479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:09.164252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 274/543] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:09.169548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbabd7-f8b5-4a16-9b6c-db1893b4fb73 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:09.177586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573d80-1756-4618-bc7a-6c2fb61353b1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:09.190464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:09.221136 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ae757-db26-49a2-80a7-bd07e76d5e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:09.222181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ae757-db26-49a2-80a7-bd07e76d5e9e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:09.222661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ae757-db26-49a2-80a7-bd07e76d5e9e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:09.223129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ae757-db26-49a2-80a7-bd07e76d5e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:09.224186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ae757-db26-49a2-80a7-bd07e76d5e9e None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=e38a311f2b614a3598f6b51b3a235178) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:09.224749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ae757-db26-49a2-80a7-bd07e76d5e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:09.226403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ae757-db26-49a2-80a7-bd07e76d5e9e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:09.232556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 270/544] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:08 2022] GET /identity/v3/role_assignments?user.id=e38a311f2b614a3598f6b51b3a235178&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:09.320754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573d80-1756-4618-bc7a-6c2fb61353b1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:09.364492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96738e8-753b-4fd4-a991-ff578bf01e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:09.365218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96738e8-753b-4fd4-a991-ff578bf01e78 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:09.365506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96738e8-753b-4fd4-a991-ff578bf01e78 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:09.365786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96738e8-753b-4fd4-a991-ff578bf01e78 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:09.366573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96738e8-753b-4fd4-a991-ff578bf01e78 None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:09.366888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96738e8-753b-4fd4-a991-ff578bf01e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:09.368330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96738e8-753b-4fd4-a991-ff578bf01e78 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:09.381237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 275/545] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:09.384985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:09.387830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ae757-db26-49a2-80a7-bd07e76d5e9e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:09.530385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ae757-db26-49a2-80a7-bd07e76d5e9e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:09.581001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa917f-7614-4994-afd9-02f0e871abe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:09.581806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa917f-7614-4994-afd9-02f0e871abe2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:09.582085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa917f-7614-4994-afd9-02f0e871abe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:09.582360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa917f-7614-4994-afd9-02f0e871abe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:09.600679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07f5bd-a492-4ae1-b4fe-e9877a4a7364 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:09.600975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07f5bd-a492-4ae1-b4fe-e9877a4a7364 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:09.601265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07f5bd-a492-4ae1-b4fe-e9877a4a7364 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:09.602930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 276/546] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:09.603620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa917f-7614-4994-afd9-02f0e871abe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:09.605150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa917f-7614-4994-afd9-02f0e871abe2 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:09.617229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639ed7b-974c-4df7-85df-d86a35545a5e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:09.617511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639ed7b-974c-4df7-85df-d86a35545a5e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:09.617791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639ed7b-974c-4df7-85df-d86a35545a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-03aa917f-7614-4994-afd9-02f0e871abe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:09.627129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:09.628153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:09.628992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:09.628992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:09.628992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:09.628992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:09.628992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 271/547] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:09.634427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa917f-7614-4994-afd9-02f0e871abe2 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:09.649390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d639ed7b-974c-4df7-85df-d86a35545a5e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:09.650721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d639ed7b-974c-4df7-85df-d86a35545a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:09.680954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:09.778308 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 277/548] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:09.793217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa917f-7614-4994-afd9-02f0e871abe2 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:09.832347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9049-2654-4111-a464-171c9bcae7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:09.833057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9049-2654-4111-a464-171c9bcae7ad None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:09.833346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9049-2654-4111-a464-171c9bcae7ad None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:09.833627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9049-2654-4111-a464-171c9bcae7ad None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:09.833726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639ed7b-974c-4df7-85df-d86a35545a5e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:09.834361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd9049-2654-4111-a464-171c9bcae7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:09.834677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd9049-2654-4111-a464-171c9bcae7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:09.836282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd9049-2654-4111-a464-171c9bcae7ad None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:09.851017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 272/549] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:09.857299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9049-2654-4111-a464-171c9bcae7ad None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:10.000957 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:10.003824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9049-2654-4111-a464-171c9bcae7ad None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:10.045354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639ed7b-974c-4df7-85df-d86a35545a5e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:10.073191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8201-4f37-42f2-9b16-85493f54dd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:10.074004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8201-4f37-42f2-9b16-85493f54dd33 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.074285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8201-4f37-42f2-9b16-85493f54dd33 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.074561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8201-4f37-42f2-9b16-85493f54dd33 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.088401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea8201-4f37-42f2-9b16-85493f54dd33 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:10.089900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea8201-4f37-42f2-9b16-85493f54dd33 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:10.108980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:10.116534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741108f-0a2c-40e2-b725-574e6b431c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:10.117353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741108f-0a2c-40e2-b725-574e6b431c34 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.117636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741108f-0a2c-40e2-b725-574e6b431c34 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.117912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741108f-0a2c-40e2-b725-574e6b431c34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-87ea8201-4f37-42f2-9b16-85493f54dd33 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:10.120679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:10.122093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:10.123333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 273/550] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:10.131860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8201-4f37-42f2-9b16-85493f54dd33 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:10.140683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741108f-0a2c-40e2-b725-574e6b431c34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-8741108f-0a2c-40e2-b725-574e6b431c34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:10.153442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:10.154365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:10.155142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:10.157479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 278/551] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:10.162684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741108f-0a2c-40e2-b725-574e6b431c34 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:10.260646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8201-4f37-42f2-9b16-85493f54dd33 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:10.280431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741108f-0a2c-40e2-b725-574e6b431c34 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:10.303191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c5a8e-2230-47c6-8c06-1d3e6acb36af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:10.304255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c5a8e-2230-47c6-8c06-1d3e6acb36af None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.304769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c5a8e-2230-47c6-8c06-1d3e6acb36af None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.305235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c5a8e-2230-47c6-8c06-1d3e6acb36af None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.306270 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c5a8e-2230-47c6-8c06-1d3e6acb36af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:10.306774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c5a8e-2230-47c6-8c06-1d3e6acb36af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:10.308409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c5a8e-2230-47c6-8c06-1d3e6acb36af None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:10.325765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1b2df-8acb-4ed7-93d7-713ec52571cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:10.326426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1b2df-8acb-4ed7-93d7-713ec52571cb None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.326707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1b2df-8acb-4ed7-93d7-713ec52571cb None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.326984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1b2df-8acb-4ed7-93d7-713ec52571cb None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.328360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1b2df-8acb-4ed7-93d7-713ec52571cb None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:10.334896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1b2df-8acb-4ed7-93d7-713ec52571cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:10.336345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1b2df-8acb-4ed7-93d7-713ec52571cb None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:10.341355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 274/552] 10.222.0.47 () {56 vars in 1135 bytes} [Fri Jun 24 23:36:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:10.348383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c5a8e-2230-47c6-8c06-1d3e6acb36af None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:10.360556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 279/553] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:10.371995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1b2df-8acb-4ed7-93d7-713ec52571cb None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:10.384996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:10.489961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:10.508295 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c5a8e-2230-47c6-8c06-1d3e6acb36af None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:10.565870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2577f2-282b-48a6-98cb-82d998b1d54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:10.566461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2577f2-282b-48a6-98cb-82d998b1d54c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.566753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2577f2-282b-48a6-98cb-82d998b1d54c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.567029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2577f2-282b-48a6-98cb-82d998b1d54c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.597805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1b2df-8acb-4ed7-93d7-713ec52571cb None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:10.640177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fdf43-424e-448e-b91e-5fa4752f3043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:10.642579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fdf43-424e-448e-b91e-5fa4752f3043 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.642864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fdf43-424e-448e-b91e-5fa4752f3043 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.643147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fdf43-424e-448e-b91e-5fa4752f3043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.669658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fdf43-424e-448e-b91e-5fa4752f3043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:10.671119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fdf43-424e-448e-b91e-5fa4752f3043 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-514fdf43-424e-448e-b91e-5fa4752f3043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:10.701337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:10.702269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:10.703653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:10.703653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:10.703653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:10.703653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:10.703653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 280/554] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:10.705643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b42ae-d8f7-477f-8bb0-283b481e04b8 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.705932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b42ae-d8f7-477f-8bb0-283b481e04b8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.706211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b42ae-d8f7-477f-8bb0-283b481e04b8 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.711471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2577f2-282b-48a6-98cb-82d998b1d54c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:10.711800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 281/555] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:10.712848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b42ae-d8f7-477f-8bb0-283b481e04b8 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:10.713137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2577f2-282b-48a6-98cb-82d998b1d54c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:10.739619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:10.842970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b42ae-d8f7-477f-8bb0-283b481e04b8 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:10.845499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:10.848608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 275/556] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:10.853992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829cd72-445b-4b25-827c-3c8234a10533 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.855185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829cd72-445b-4b25-827c-3c8234a10533 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.855657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829cd72-445b-4b25-827c-3c8234a10533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.878939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c829cd72-445b-4b25-827c-3c8234a10533 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:10.880482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c829cd72-445b-4b25-827c-3c8234a10533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:10.880621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97eb71-6431-438a-8075-5b622f2e6fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:10.881239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97eb71-6431-438a-8075-5b622f2e6fca None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.881521 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97eb71-6431-438a-8075-5b622f2e6fca None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.881798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97eb71-6431-438a-8075-5b622f2e6fca None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.882522 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97eb71-6431-438a-8075-5b622f2e6fca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:10.882831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97eb71-6431-438a-8075-5b622f2e6fca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:10.884189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97eb71-6431-438a-8075-5b622f2e6fca None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:10.895663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 282/557] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:10.896932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03a82f-0d13-429c-ab7f-308f32035e03 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:10.897223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03a82f-0d13-429c-ab7f-308f32035e03 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:10.897505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03a82f-0d13-429c-ab7f-308f32035e03 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:10.902841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 283/558] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:10.902841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03a82f-0d13-429c-ab7f-308f32035e03 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:10.969391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 276/559] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:10.970236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829cd72-445b-4b25-827c-3c8234a10533 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.014741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03a82f-0d13-429c-ab7f-308f32035e03 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.025350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:11.055300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc348be1-854c-4d35-b8c9-7c7eb202c8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.055675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc348be1-854c-4d35-b8c9-7c7eb202c8e0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.055956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc348be1-854c-4d35-b8c9-7c7eb202c8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.056448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc348be1-854c-4d35-b8c9-7c7eb202c8e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.066796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc348be1-854c-4d35-b8c9-7c7eb202c8e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:11.068226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc348be1-854c-4d35-b8c9-7c7eb202c8e0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-fc348be1-854c-4d35-b8c9-7c7eb202c8e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:11.085289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:11.086194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:11.087050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:11.087050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:11.087050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:11.087050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:11.087050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 284/560] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:11.087631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eb045-393d-4825-a30c-cea9ca942b20 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.087914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eb045-393d-4825-a30c-cea9ca942b20 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.088188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eb045-393d-4825-a30c-cea9ca942b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.104785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829cd72-445b-4b25-827c-3c8234a10533 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.106293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-790eb045-393d-4825-a30c-cea9ca942b20 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:11.107865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-790eb045-393d-4825-a30c-cea9ca942b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:11.117900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:11.145103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01500c2-0fbd-49f3-8a34-10e3af116f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.146000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01500c2-0fbd-49f3-8a34-10e3af116f72 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.146289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01500c2-0fbd-49f3-8a34-10e3af116f72 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.146565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01500c2-0fbd-49f3-8a34-10e3af116f72 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.154584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01500c2-0fbd-49f3-8a34-10e3af116f72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:11.156653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01500c2-0fbd-49f3-8a34-10e3af116f72 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-e01500c2-0fbd-49f3-8a34-10e3af116f72 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:11.171126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:11.172463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 23:36:11.173619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:11.176884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 277/561] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:11.177716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01500c2-0fbd-49f3-8a34-10e3af116f72 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.181966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 285/562] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:11.182436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eb045-393d-4825-a30c-cea9ca942b20 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.289698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eb045-393d-4825-a30c-cea9ca942b20 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.292892 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01500c2-0fbd-49f3-8a34-10e3af116f72 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.294424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:11.330306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43f879-0edb-41ee-9048-92d70e9151fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.331111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43f879-0edb-41ee-9048-92d70e9151fa None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.331398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43f879-0edb-41ee-9048-92d70e9151fa None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.331674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43f879-0edb-41ee-9048-92d70e9151fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.333309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19ccc3-b256-4780-9114-842ae3288ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.333979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19ccc3-b256-4780-9114-842ae3288ba6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.334258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19ccc3-b256-4780-9114-842ae3288ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.334540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19ccc3-b256-4780-9114-842ae3288ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.335273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19ccc3-b256-4780-9114-842ae3288ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:11.335577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19ccc3-b256-4780-9114-842ae3288ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:11.336981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19ccc3-b256-4780-9114-842ae3288ba6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:11.344671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43f879-0edb-41ee-9048-92d70e9151fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:11.349013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 286/563] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:11.349819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19ccc3-b256-4780-9114-842ae3288ba6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-7c43f879-0edb-41ee-9048-92d70e9151fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:11.354031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:11.355071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:11.355922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:11.357149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 278/564] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:11.357468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43f879-0edb-41ee-9048-92d70e9151fa None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.363437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:11.458611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43f879-0edb-41ee-9048-92d70e9151fa None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.471831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19ccc3-b256-4780-9114-842ae3288ba6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.493157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24f29d-fa2f-47f8-a899-eb2fb62e81c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.493961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24f29d-fa2f-47f8-a899-eb2fb62e81c5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.494244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24f29d-fa2f-47f8-a899-eb2fb62e81c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.494524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24f29d-fa2f-47f8-a899-eb2fb62e81c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.504573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24f29d-fa2f-47f8-a899-eb2fb62e81c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:11.507354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f18ee7-f99c-4d47-959d-9f5c27b0c611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.508189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f18ee7-f99c-4d47-959d-9f5c27b0c611 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.508645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f18ee7-f99c-4d47-959d-9f5c27b0c611 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.509056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f18ee7-f99c-4d47-959d-9f5c27b0c611 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.510031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f18ee7-f99c-4d47-959d-9f5c27b0c611 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:11.510510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f18ee7-f99c-4d47-959d-9f5c27b0c611 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:11.512065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f18ee7-f99c-4d47-959d-9f5c27b0c611 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-4b24f29d-fa2f-47f8-a899-eb2fb62e81c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:11.512230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:11.513303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:11.514143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 279/565] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:11.514800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24f29d-fa2f-47f8-a899-eb2fb62e81c5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.536438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 287/566] 10.222.0.47 () {56 vars in 1137 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:11.537447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f18ee7-f99c-4d47-959d-9f5c27b0c611 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.548997 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:11.604140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:11.634697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24f29d-fa2f-47f8-a899-eb2fb62e81c5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.649032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f18ee7-f99c-4d47-959d-9f5c27b0c611 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.669420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c1306-7b3f-4ef1-937f-8b944f1b13db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.670239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c1306-7b3f-4ef1-937f-8b944f1b13db None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.670533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c1306-7b3f-4ef1-937f-8b944f1b13db None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.670809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c1306-7b3f-4ef1-937f-8b944f1b13db None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.677564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c1306-7b3f-4ef1-937f-8b944f1b13db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:11.679014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c1306-7b3f-4ef1-937f-8b944f1b13db None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:11.683653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806f86b-4e98-4e0b-8d46-a5ce0b1df193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.684317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806f86b-4e98-4e0b-8d46-a5ce0b1df193 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.684618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806f86b-4e98-4e0b-8d46-a5ce0b1df193 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.684902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806f86b-4e98-4e0b-8d46-a5ce0b1df193 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.685643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806f86b-4e98-4e0b-8d46-a5ce0b1df193 None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:11.685945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806f86b-4e98-4e0b-8d46-a5ce0b1df193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:11.687795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f806f86b-4e98-4e0b-8d46-a5ce0b1df193 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-793c1306-7b3f-4ef1-937f-8b944f1b13db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:11.693476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:11.694501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:11.695404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:11.695404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:11.695404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:11.695404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:11.695404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:11.695404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:11.695404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:11.695404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:11.695404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 280/567] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:11.695877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c1306-7b3f-4ef1-937f-8b944f1b13db None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.699803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 288/568] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:11.700676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806f86b-4e98-4e0b-8d46-a5ce0b1df193 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.776026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:11.794631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c1306-7b3f-4ef1-937f-8b944f1b13db None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.799791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806f86b-4e98-4e0b-8d46-a5ce0b1df193 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.822278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:11.828644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d82c18-8917-43ad-a6ef-45dd566458b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.829307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d82c18-8917-43ad-a6ef-45dd566458b7 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.829586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d82c18-8917-43ad-a6ef-45dd566458b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.829858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d82c18-8917-43ad-a6ef-45dd566458b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.830597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d82c18-8917-43ad-a6ef-45dd566458b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:11.830899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d82c18-8917-43ad-a6ef-45dd566458b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:11.832259 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d82c18-8917-43ad-a6ef-45dd566458b7 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:11.833118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d550c0-cd9d-4f90-87e9-2bc134058d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.833906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d550c0-cd9d-4f90-87e9-2bc134058d84 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.834189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d550c0-cd9d-4f90-87e9-2bc134058d84 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.834469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d550c0-cd9d-4f90-87e9-2bc134058d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.840171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 281/569] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:11.841478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b475195-1df5-41d9-9fb7-6a016775e1a3 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.841769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b475195-1df5-41d9-9fb7-6a016775e1a3 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.842048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b475195-1df5-41d9-9fb7-6a016775e1a3 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.843248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 282/570] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:11.844937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b475195-1df5-41d9-9fb7-6a016775e1a3 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.850680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d550c0-cd9d-4f90-87e9-2bc134058d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:11.852145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d550c0-cd9d-4f90-87e9-2bc134058d84 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-52d550c0-cd9d-4f90-87e9-2bc134058d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:11.869005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:11.869866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:11.870722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:11.870722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:11.870722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:11.870722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:11.870722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 289/571] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:11.871043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d550c0-cd9d-4f90-87e9-2bc134058d84 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:11.951813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b475195-1df5-41d9-9fb7-6a016775e1a3 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.972936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d550c0-cd9d-4f90-87e9-2bc134058d84 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:11.985871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4435737-5a45-4d49-a250-cff2d754f0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:11.986528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4435737-5a45-4d49-a250-cff2d754f0a1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:11.986810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4435737-5a45-4d49-a250-cff2d754f0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:11.987089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4435737-5a45-4d49-a250-cff2d754f0a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:11.987860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4435737-5a45-4d49-a250-cff2d754f0a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:11.988169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4435737-5a45-4d49-a250-cff2d754f0a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:11.989628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4435737-5a45-4d49-a250-cff2d754f0a1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.001241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 283/572] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:12.002013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4435737-5a45-4d49-a250-cff2d754f0a1 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.005341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:12.012105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af052426-e19a-45e4-874a-5e80522a011b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.012948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052426-e19a-45e4-874a-5e80522a011b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.013236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052426-e19a-45e4-874a-5e80522a011b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.013537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052426-e19a-45e4-874a-5e80522a011b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.024846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af052426-e19a-45e4-874a-5e80522a011b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.026283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af052426-e19a-45e4-874a-5e80522a011b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-af052426-e19a-45e4-874a-5e80522a011b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:12.044524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:12.045382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:12.046214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:12.046214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:12.046214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:12.046214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:12.046214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 290/573] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:12.047452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90ceaf-b48c-4165-abf0-37c1b374be9b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.047713 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90ceaf-b48c-4165-abf0-37c1b374be9b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.047990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90ceaf-b48c-4165-abf0-37c1b374be9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.064136 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-1f90ceaf-b48c-4165-abf0-37c1b374be9b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:12.065348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1f90ceaf-b48c-4165-abf0-37c1b374be9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:12.087556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:12.103767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4435737-5a45-4d49-a250-cff2d754f0a1 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.131909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 291/574] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:12.133052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90ceaf-b48c-4165-abf0-37c1b374be9b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.144042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c1608-8e6d-44bc-b136-2b2ca434cafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.144870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c1608-8e6d-44bc-b136-2b2ca434cafe None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.145160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c1608-8e6d-44bc-b136-2b2ca434cafe None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.145439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c1608-8e6d-44bc-b136-2b2ca434cafe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.156050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c1608-8e6d-44bc-b136-2b2ca434cafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.157328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c1608-8e6d-44bc-b136-2b2ca434cafe None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-2f3c1608-8e6d-44bc-b136-2b2ca434cafe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:12.175079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:12.175984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:12.176904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:12.176904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:12.176904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:12.176904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:12.176904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 284/575] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:12.177419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c1608-8e6d-44bc-b136-2b2ca434cafe None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.235412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:12.238966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90ceaf-b48c-4165-abf0-37c1b374be9b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.273672 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97bde6-d4c9-4bb8-8586-a8715b77f63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.274324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97bde6-d4c9-4bb8-8586-a8715b77f63b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.274611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97bde6-d4c9-4bb8-8586-a8715b77f63b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.274880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97bde6-d4c9-4bb8-8586-a8715b77f63b None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.275600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97bde6-d4c9-4bb8-8586-a8715b77f63b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:12.275901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97bde6-d4c9-4bb8-8586-a8715b77f63b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.277291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97bde6-d4c9-4bb8-8586-a8715b77f63b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.278314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c1608-8e6d-44bc-b136-2b2ca434cafe None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.290269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 292/576] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:12.291084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97bde6-d4c9-4bb8-8586-a8715b77f63b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.309785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:12.314054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe0e54-fecb-40c3-9796-d06b8d103a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.314708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe0e54-fecb-40c3-9796-d06b8d103a7e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.314990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe0e54-fecb-40c3-9796-d06b8d103a7e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.315259 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe0e54-fecb-40c3-9796-d06b8d103a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.316090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe0e54-fecb-40c3-9796-d06b8d103a7e None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=3dd1e02af2374224b968c2644dd0c813) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:12.316398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe0e54-fecb-40c3-9796-d06b8d103a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.317878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe0e54-fecb-40c3-9796-d06b8d103a7e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.323231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 285/577] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/role_assignments?user.id=3dd1e02af2374224b968c2644dd0c813&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:12.323999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe0e54-fecb-40c3-9796-d06b8d103a7e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.400604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97bde6-d4c9-4bb8-8586-a8715b77f63b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.439987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc452e1-2413-40d3-8c79-574a0142e0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.440674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe0e54-fecb-40c3-9796-d06b8d103a7e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.440794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc452e1-2413-40d3-8c79-574a0142e0b2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.440970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc452e1-2413-40d3-8c79-574a0142e0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.441250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc452e1-2413-40d3-8c79-574a0142e0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.441994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc452e1-2413-40d3-8c79-574a0142e0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:12.442301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc452e1-2413-40d3-8c79-574a0142e0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.443697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc452e1-2413-40d3-8c79-574a0142e0b2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.457842 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 293/578] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:12.458600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc452e1-2413-40d3-8c79-574a0142e0b2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.469483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:12.485526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00521d-0645-47e7-88fb-6220d0f21d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.486261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00521d-0645-47e7-88fb-6220d0f21d18 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.486543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00521d-0645-47e7-88fb-6220d0f21d18 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.486823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00521d-0645-47e7-88fb-6220d0f21d18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.497881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00521d-0645-47e7-88fb-6220d0f21d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-cc00521d-0645-47e7-88fb-6220d0f21d18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:12.506117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:12.507022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:12.507857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 286/579] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:12.512818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00521d-0645-47e7-88fb-6220d0f21d18 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.553165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:12.577431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc452e1-2413-40d3-8c79-574a0142e0b2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.613399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50c1c9-b76e-4b04-8d16-ed4d35380f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.614066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50c1c9-b76e-4b04-8d16-ed4d35380f66 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.614348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50c1c9-b76e-4b04-8d16-ed4d35380f66 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.614622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50c1c9-b76e-4b04-8d16-ed4d35380f66 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.615339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50c1c9-b76e-4b04-8d16-ed4d35380f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:12.615646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50c1c9-b76e-4b04-8d16-ed4d35380f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.617328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50c1c9-b76e-4b04-8d16-ed4d35380f66 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.618987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00521d-0645-47e7-88fb-6220d0f21d18 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.631437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 294/580] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:12.632256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50c1c9-b76e-4b04-8d16-ed4d35380f66 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.661684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78b43e-e509-450e-a33a-b54178cb34a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.662486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78b43e-e509-450e-a33a-b54178cb34a1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.662771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78b43e-e509-450e-a33a-b54178cb34a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.663054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78b43e-e509-450e-a33a-b54178cb34a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.669909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78b43e-e509-450e-a33a-b54178cb34a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.671352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78b43e-e509-450e-a33a-b54178cb34a1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-2a78b43e-e509-450e-a33a-b54178cb34a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:12.686658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:12.688268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:12.689274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:12.689274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:12.689274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:12.689274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:12.689274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:12.689274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:12.689274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:12.689274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:12.689274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 287/581] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:12.689747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78b43e-e509-450e-a33a-b54178cb34a1 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.735152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:12.751124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50c1c9-b76e-4b04-8d16-ed4d35380f66 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.790231 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2d6a3-6574-471d-85c3-0ea88a073fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.791119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2d6a3-6574-471d-85c3-0ea88a073fe1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.791471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2d6a3-6574-471d-85c3-0ea88a073fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.791821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2d6a3-6574-471d-85c3-0ea88a073fe1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.799324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78b43e-e509-450e-a33a-b54178cb34a1 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.800546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2d6a3-6574-471d-85c3-0ea88a073fe1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.802367 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2d6a3-6574-471d-85c3-0ea88a073fe1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.803958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ded2d6a3-6574-471d-85c3-0ea88a073fe1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:12.818239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:12.819180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 295/582] 10.222.0.47 () {56 vars in 1093 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:12.825486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2d6a3-6574-471d-85c3-0ea88a073fe1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.840928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bb563-cdcf-48b7-bf4c-0665c9c0ac9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.841683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb563-cdcf-48b7-bf4c-0665c9c0ac9f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.842021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb563-cdcf-48b7-bf4c-0665c9c0ac9f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.842296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb563-cdcf-48b7-bf4c-0665c9c0ac9f None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.843044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bb563-cdcf-48b7-bf4c-0665c9c0ac9f None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:12.844273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bb563-cdcf-48b7-bf4c-0665c9c0ac9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.845785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bb563-cdcf-48b7-bf4c-0665c9c0ac9f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.855023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 288/583] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:12.860527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bb563-cdcf-48b7-bf4c-0665c9c0ac9f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:12.926814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2d6a3-6574-471d-85c3-0ea88a073fe1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.967375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e36e93-be7a-4633-aa00-ce7cd2927f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:12.968172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e36e93-be7a-4633-aa00-ce7cd2927f2f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:12.968622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e36e93-be7a-4633-aa00-ce7cd2927f2f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:12.968893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e36e93-be7a-4633-aa00-ce7cd2927f2f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:12.971840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bb563-cdcf-48b7-bf4c-0665c9c0ac9f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:12.973638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:12.977839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e36e93-be7a-4633-aa00-ce7cd2927f2f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:12.979290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e36e93-be7a-4633-aa00-ce7cd2927f2f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-14e36e93-be7a-4633-aa00-ce7cd2927f2f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:12.993839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:12.994805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:12.995709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 296/584] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:12.997173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e36e93-be7a-4633-aa00-ce7cd2927f2f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.016154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20548bf3-7fb1-410e-a541-79eb89ad5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.016874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20548bf3-7fb1-410e-a541-79eb89ad5809 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.017161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20548bf3-7fb1-410e-a541-79eb89ad5809 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.017439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20548bf3-7fb1-410e-a541-79eb89ad5809 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.018235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20548bf3-7fb1-410e-a541-79eb89ad5809 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:13.018550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20548bf3-7fb1-410e-a541-79eb89ad5809 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.020061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20548bf3-7fb1-410e-a541-79eb89ad5809 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.033237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 289/585] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:13.034461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20548bf3-7fb1-410e-a541-79eb89ad5809 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.061713 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:13.108662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e36e93-be7a-4633-aa00-ce7cd2927f2f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.141755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20548bf3-7fb1-410e-a541-79eb89ad5809 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.144514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeefc83-ea03-4f86-95ff-bb645e42e1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.145225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeefc83-ea03-4f86-95ff-bb645e42e1b5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.145515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeefc83-ea03-4f86-95ff-bb645e42e1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.145795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeefc83-ea03-4f86-95ff-bb645e42e1b5 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.146604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeefc83-ea03-4f86-95ff-bb645e42e1b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:13.146907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeefc83-ea03-4f86-95ff-bb645e42e1b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.148625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeefc83-ea03-4f86-95ff-bb645e42e1b5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.168893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 297/586] 10.222.0.47 () {56 vars in 1133 bytes} [Fri Jun 24 23:36:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:13.169724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeefc83-ea03-4f86-95ff-bb645e42e1b5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.185382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999368c-3683-4316-ba39-e93dae61e6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.186176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999368c-3683-4316-ba39-e93dae61e6f1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.186457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999368c-3683-4316-ba39-e93dae61e6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.186727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999368c-3683-4316-ba39-e93dae61e6f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.198510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999368c-3683-4316-ba39-e93dae61e6f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.200157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999368c-3683-4316-ba39-e93dae61e6f1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-f999368c-3683-4316-ba39-e93dae61e6f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:13.217761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:13.219274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:13.220149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:13.220149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:13.220149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:13.220149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:13.220149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 290/587] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:13.220880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999368c-3683-4316-ba39-e93dae61e6f1 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.237364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:13.279994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeefc83-ea03-4f86-95ff-bb645e42e1b5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.298543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:13.316273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc1471-dfc0-4f6d-b07a-d99bfa1f5c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.317001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc1471-dfc0-4f6d-b07a-d99bfa1f5c42 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.317291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc1471-dfc0-4f6d-b07a-d99bfa1f5c42 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.317570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc1471-dfc0-4f6d-b07a-d99bfa1f5c42 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.318415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc1471-dfc0-4f6d-b07a-d99bfa1f5c42 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:13.318727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc1471-dfc0-4f6d-b07a-d99bfa1f5c42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.320144 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc1471-dfc0-4f6d-b07a-d99bfa1f5c42 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.333049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999368c-3683-4316-ba39-e93dae61e6f1 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.337458 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 298/588] 10.222.0.47 () {56 vars in 1137 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:13.338082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc1471-dfc0-4f6d-b07a-d99bfa1f5c42 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.380412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69b45f-4148-4ce7-86bd-6e0a0a4a4b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.381336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69b45f-4148-4ce7-86bd-6e0a0a4a4b3c None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.381619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69b45f-4148-4ce7-86bd-6e0a0a4a4b3c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.381896 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69b45f-4148-4ce7-86bd-6e0a0a4a4b3c None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/71e5ef9dc32f494791ba4addb5151232/roles/147f73b08d294eae9859df32606b7d69` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.402037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69b45f-4148-4ce7-86bd-6e0a0a4a4b3c None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=71e5ef9dc32f494791ba4addb5151232, role_id=147f73b08d294eae9859df32606b7d69)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.403501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69b45f-4148-4ce7-86bd-6e0a0a4a4b3c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.428617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 291/589] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:36:13 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/71e5ef9dc32f494791ba4addb5151232/roles/147f73b08d294eae9859df32606b7d69 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:36:13.428617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69b45f-4148-4ce7-86bd-6e0a0a4a4b3c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.475911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc1471-dfc0-4f6d-b07a-d99bfa1f5c42 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.513350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509a7e4-c570-48b1-884d-2e0f6fed4869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.514192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:13.514335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509a7e4-c570-48b1-884d-2e0f6fed4869 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.515045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509a7e4-c570-48b1-884d-2e0f6fed4869 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.515045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509a7e4-c570-48b1-884d-2e0f6fed4869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.532268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509a7e4-c570-48b1-884d-2e0f6fed4869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.534070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509a7e4-c570-48b1-884d-2e0f6fed4869 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-9509a7e4-c570-48b1-884d-2e0f6fed4869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:13.556301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:13.557324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:13.558265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:13.558265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:13.558265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:13.558265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:13.558265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 299/590] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:13.559340 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509a7e4-c570-48b1-884d-2e0f6fed4869 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.574106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69b45f-4148-4ce7-86bd-6e0a0a4a4b3c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.598797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:13.617445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898537dc-3b2a-48bd-8924-3cff91ce0ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.618317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898537dc-3b2a-48bd-8924-3cff91ce0ed1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.619021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898537dc-3b2a-48bd-8924-3cff91ce0ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.619021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898537dc-3b2a-48bd-8924-3cff91ce0ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.619899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898537dc-3b2a-48bd-8924-3cff91ce0ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:13.620239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898537dc-3b2a-48bd-8924-3cff91ce0ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.621958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898537dc-3b2a-48bd-8924-3cff91ce0ed1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.634493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 292/591] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:13.635367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898537dc-3b2a-48bd-8924-3cff91ce0ed1 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.689277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509a7e4-c570-48b1-884d-2e0f6fed4869 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.731496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ef4a-59d6-4aa1-9d53-a199dd26b005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.732690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ef4a-59d6-4aa1-9d53-a199dd26b005 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.733154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ef4a-59d6-4aa1-9d53-a199dd26b005 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.733634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ef4a-59d6-4aa1-9d53-a199dd26b005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.746098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3ef4a-59d6-4aa1-9d53-a199dd26b005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.747707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898537dc-3b2a-48bd-8924-3cff91ce0ed1 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.748058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3ef4a-59d6-4aa1-9d53-a199dd26b005 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-bee3ef4a-59d6-4aa1-9d53-a199dd26b005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:13.769456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:13.770376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:13.771234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:13.771234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:13.771234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:13.771234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:13.772356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:13.778553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 300/592] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:13.779783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ef4a-59d6-4aa1-9d53-a199dd26b005 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.786710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a225f7d-3bfd-422e-831b-dde6a413e550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.787162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a225f7d-3bfd-422e-831b-dde6a413e550 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.787442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a225f7d-3bfd-422e-831b-dde6a413e550 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.787727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a225f7d-3bfd-422e-831b-dde6a413e550 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.788452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a225f7d-3bfd-422e-831b-dde6a413e550 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:13.788807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a225f7d-3bfd-422e-831b-dde6a413e550 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.790182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a225f7d-3bfd-422e-831b-dde6a413e550 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.807310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 293/593] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:13.808213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a225f7d-3bfd-422e-831b-dde6a413e550 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.873678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:13.899634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ef4a-59d6-4aa1-9d53-a199dd26b005 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.912139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a225f7d-3bfd-422e-831b-dde6a413e550 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:13.941658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120f2f6-9320-4a99-b5ca-417cd1aab8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.942756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120f2f6-9320-4a99-b5ca-417cd1aab8d1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.943285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120f2f6-9320-4a99-b5ca-417cd1aab8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.943707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120f2f6-9320-4a99-b5ca-417cd1aab8d1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.951561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9a76-c1fe-4e64-a568-d87b6a2d444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:13.952221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9a76-c1fe-4e64-a568-d87b6a2d444f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:13.952498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9a76-c1fe-4e64-a568-d87b6a2d444f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:13.952686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120f2f6-9320-4a99-b5ca-417cd1aab8d1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.952901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc9a76-c1fe-4e64-a568-d87b6a2d444f None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:13.953579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc9a76-c1fe-4e64-a568-d87b6a2d444f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:13.953885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc9a76-c1fe-4e64-a568-d87b6a2d444f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:13.956642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120f2f6-9320-4a99-b5ca-417cd1aab8d1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.956767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc9a76-c1fe-4e64-a568-d87b6a2d444f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:13.967485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 294/594] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:13.969068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9a76-c1fe-4e64-a568-d87b6a2d444f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-2120f2f6-9320-4a99-b5ca-417cd1aab8d1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:13.975622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:13.976550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:13.977497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 301/595] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:13.978075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120f2f6-9320-4a99-b5ca-417cd1aab8d1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:14.020647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:14.088014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc9a76-c1fe-4e64-a568-d87b6a2d444f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:14.119356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120f2f6-9320-4a99-b5ca-417cd1aab8d1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:14.131966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9cf07-bbb0-435f-a7cb-412ae39de69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:14.133720 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9cf07-bbb0-435f-a7cb-412ae39de69f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.134152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9cf07-bbb0-435f-a7cb-412ae39de69f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.134571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9cf07-bbb0-435f-a7cb-412ae39de69f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.145083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9cf07-bbb0-435f-a7cb-412ae39de69f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:14.147625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9cf07-bbb0-435f-a7cb-412ae39de69f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:14.158344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394ae03-dd7e-4392-a332-0bef42bf4478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:14.159265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394ae03-dd7e-4392-a332-0bef42bf4478 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.159614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394ae03-dd7e-4392-a332-0bef42bf4478 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.159962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394ae03-dd7e-4392-a332-0bef42bf4478 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-fbf9cf07-bbb0-435f-a7cb-412ae39de69f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:14.165952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:14.166986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:14.168030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:14.168030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:14.168030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:14.168030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:14.168030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:14.168030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:14.168030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:14.168030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:14.175180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 295/596] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:14.175180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9cf07-bbb0-435f-a7cb-412ae39de69f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:14.175908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394ae03-dd7e-4392-a332-0bef42bf4478 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:14.177270 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394ae03-dd7e-4392-a332-0bef42bf4478 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:14.179243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-8394ae03-dd7e-4392-a332-0bef42bf4478 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:14.193616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:14.194556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:14.195412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:14.195412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:14.195412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:14.195412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:14.195412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:14.195412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:14.195412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:14.195412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:14.202808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 302/597] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:14.203821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394ae03-dd7e-4392-a332-0bef42bf4478 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:14.315264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394ae03-dd7e-4392-a332-0bef42bf4478 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:14.318084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9cf07-bbb0-435f-a7cb-412ae39de69f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:14.321280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:14.356700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0001d1-6dba-4f4a-aa90-058cf2b0b995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:14.357529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0001d1-6dba-4f4a-aa90-058cf2b0b995 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.357911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0001d1-6dba-4f4a-aa90-058cf2b0b995 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.358401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0001d1-6dba-4f4a-aa90-058cf2b0b995 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.359325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0001d1-6dba-4f4a-aa90-058cf2b0b995 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:14.359735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0001d1-6dba-4f4a-aa90-058cf2b0b995 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:14.361612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0001d1-6dba-4f4a-aa90-058cf2b0b995 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:14.364154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b78727-c9af-4bfb-900b-30fbe66f447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:14.365982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b78727-c9af-4bfb-900b-30fbe66f447f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.366413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b78727-c9af-4bfb-900b-30fbe66f447f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.366833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b78727-c9af-4bfb-900b-30fbe66f447f None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.367765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b78727-c9af-4bfb-900b-30fbe66f447f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:14.368234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b78727-c9af-4bfb-900b-30fbe66f447f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:14.370200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b78727-c9af-4bfb-900b-30fbe66f447f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:14.381891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 296/598] 10.222.0.47 () {56 vars in 1139 bytes} [Fri Jun 24 23:36:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:14.383702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0001d1-6dba-4f4a-aa90-058cf2b0b995 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:14.397030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 303/599] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:14.398224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b78727-c9af-4bfb-900b-30fbe66f447f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:14.457967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:14.495496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0001d1-6dba-4f4a-aa90-058cf2b0b995 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:14.501343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b78727-c9af-4bfb-900b-30fbe66f447f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:14.531965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abcdba-73f7-426b-985c-7bc15e90c8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:14.532645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abcdba-73f7-426b-985c-7bc15e90c8b1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.532926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abcdba-73f7-426b-985c-7bc15e90c8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.533204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abcdba-73f7-426b-985c-7bc15e90c8b1 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.534013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abcdba-73f7-426b-985c-7bc15e90c8b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:14.534315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abcdba-73f7-426b-985c-7bc15e90c8b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:14.535722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abcdba-73f7-426b-985c-7bc15e90c8b1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:14.538934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ab0fe-2805-4abb-b2bb-2ce34e41fe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:14.539971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ab0fe-2805-4abb-b2bb-2ce34e41fe2f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.540429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ab0fe-2805-4abb-b2bb-2ce34e41fe2f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.540899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ab0fe-2805-4abb-b2bb-2ce34e41fe2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.547187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 297/600] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:14.548560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ba061-66da-471b-bc57-64a6ae34aeb4 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.548881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ba061-66da-471b-bc57-64a6ae34aeb4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.549159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ba061-66da-471b-bc57-64a6ae34aeb4 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.552384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 298/601] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:14.552384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ba061-66da-471b-bc57-64a6ae34aeb4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:14.556018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ab0fe-2805-4abb-b2bb-2ce34e41fe2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:14.557498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ab0fe-2805-4abb-b2bb-2ce34e41fe2f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-6f1ab0fe-2805-4abb-b2bb-2ce34e41fe2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:14.577328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:14.578408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:14.579265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:14.579265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:14.579265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:14.579265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:14.579265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 304/602] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:14.579265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ab0fe-2805-4abb-b2bb-2ce34e41fe2f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:14.584562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:14.676383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ba061-66da-471b-bc57-64a6ae34aeb4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:14.687878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ab0fe-2805-4abb-b2bb-2ce34e41fe2f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:14.698471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:14.713731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfffe4-9435-4501-8fed-ef3139ecdd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:14.714622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfffe4-9435-4501-8fed-ef3139ecdd03 None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.714904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfffe4-9435-4501-8fed-ef3139ecdd03 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.715180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfffe4-9435-4501-8fed-ef3139ecdd03 None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/feb9bec9f1894e179e24a14b81b3f23d/roles/147f73b08d294eae9859df32606b7d69` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.724412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492414b-322a-4bad-a948-788c847132ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:14.725563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492414b-322a-4bad-a948-788c847132ce None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.726025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492414b-322a-4bad-a948-788c847132ce None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.726480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492414b-322a-4bad-a948-788c847132ce None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/fb004ddd208849f6a5316c19e9a916aa/roles/147f73b08d294eae9859df32606b7d69` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.730492 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfffe4-9435-4501-8fed-ef3139ecdd03 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=feb9bec9f1894e179e24a14b81b3f23d, role_id=147f73b08d294eae9859df32606b7d69)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:14.731942 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfffe4-9435-4501-8fed-ef3139ecdd03 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:14.743448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492414b-322a-4bad-a948-788c847132ce None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=fb004ddd208849f6a5316c19e9a916aa, role_id=147f73b08d294eae9859df32606b7d69)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:14.745159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492414b-322a-4bad-a948-788c847132ce None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:14.745269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 299/603] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:36:14 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/feb9bec9f1894e179e24a14b81b3f23d/roles/147f73b08d294eae9859df32606b7d69 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:36:14.746399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee09caf-0283-43af-82a1-bc687dc861b0 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:14.746686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee09caf-0283-43af-82a1-bc687dc861b0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:14.746963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee09caf-0283-43af-82a1-bc687dc861b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:14.767700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2ee09caf-0283-43af-82a1-bc687dc861b0 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:14.769047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2ee09caf-0283-43af-82a1-bc687dc861b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:14.783471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 305/604] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:36:14 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/fb004ddd208849f6a5316c19e9a916aa/roles/147f73b08d294eae9859df32606b7d69 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:36:14.783471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492414b-322a-4bad-a948-788c847132ce None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:14.914757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 300/605] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:14.949741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee09caf-0283-43af-82a1-bc687dc861b0 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:14.952256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:14.989343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492414b-322a-4bad-a948-788c847132ce None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:15.027331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b5ba9-5115-4151-baa2-6d8f6705cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:15.028062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b5ba9-5115-4151-baa2-6d8f6705cd09 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.028288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b5ba9-5115-4151-baa2-6d8f6705cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.028564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b5ba9-5115-4151-baa2-6d8f6705cd09 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.029343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b5ba9-5115-4151-baa2-6d8f6705cd09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:15.029660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b5ba9-5115-4151-baa2-6d8f6705cd09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:15.031079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b5ba9-5115-4151-baa2-6d8f6705cd09 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:15.047271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 306/606] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:15.053422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b5ba9-5115-4151-baa2-6d8f6705cd09 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:15.081601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:15.083887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee09caf-0283-43af-82a1-bc687dc861b0 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:15.146087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0f6c1-95e5-4006-aecd-e8f5b6adc922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:15.146954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0f6c1-95e5-4006-aecd-e8f5b6adc922 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.147285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0f6c1-95e5-4006-aecd-e8f5b6adc922 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.147513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0f6c1-95e5-4006-aecd-e8f5b6adc922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.177570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0f6c1-95e5-4006-aecd-e8f5b6adc922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-05a0f6c1-95e5-4006-aecd-e8f5b6adc922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:15.185470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:15.186626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:15.187988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:15.188792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 301/607] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:15.198659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0f6c1-95e5-4006-aecd-e8f5b6adc922 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:15.199632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b5ba9-5115-4151-baa2-6d8f6705cd09 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:15.249374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8cce8-1399-4332-85af-ddc443ddcb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:15.250516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8cce8-1399-4332-85af-ddc443ddcb5e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.250981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8cce8-1399-4332-85af-ddc443ddcb5e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.251436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8cce8-1399-4332-85af-ddc443ddcb5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.259075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8cce8-1399-4332-85af-ddc443ddcb5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:15.260788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8cce8-1399-4332-85af-ddc443ddcb5e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:15.277575 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-17f8cce8-1399-4332-85af-ddc443ddcb5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:15.283838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:15.284835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:15.285685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:15.285685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:15.285685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:15.285685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:15.285685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:15.285685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:15.285685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:15.285685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:15.293621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 307/608] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:15.310082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8cce8-1399-4332-85af-ddc443ddcb5e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:15.340288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0f6c1-95e5-4006-aecd-e8f5b6adc922 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:15.403649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddd914-bc0b-42ee-8772-94d2f90707bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:15.404428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddd914-bc0b-42ee-8772-94d2f90707bc None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.405601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddd914-bc0b-42ee-8772-94d2f90707bc None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.406658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddd914-bc0b-42ee-8772-94d2f90707bc None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.407442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddd914-bc0b-42ee-8772-94d2f90707bc None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:15.407756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddd914-bc0b-42ee-8772-94d2f90707bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:15.409189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddd914-bc0b-42ee-8772-94d2f90707bc None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:15.430683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 302/609] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:15.436894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddd914-bc0b-42ee-8772-94d2f90707bc None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:15.452193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:15.475314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8cce8-1399-4332-85af-ddc443ddcb5e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:15.512914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f88922-5de9-4a32-85d8-973519ff53c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:15.513555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f88922-5de9-4a32-85d8-973519ff53c4 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.513844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f88922-5de9-4a32-85d8-973519ff53c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.514526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f88922-5de9-4a32-85d8-973519ff53c4 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.516123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f88922-5de9-4a32-85d8-973519ff53c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:15.516466 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f88922-5de9-4a32-85d8-973519ff53c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:15.518085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f88922-5de9-4a32-85d8-973519ff53c4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:15.532467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 308/610] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:15.538628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f88922-5de9-4a32-85d8-973519ff53c4 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:15.625154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddd914-bc0b-42ee-8772-94d2f90707bc None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:15.650991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:15.663581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e7b0-d472-4f23-8329-a3b992f27430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:15.664383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e7b0-d472-4f23-8329-a3b992f27430 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.664741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e7b0-d472-4f23-8329-a3b992f27430 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.665122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e7b0-d472-4f23-8329-a3b992f27430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.670741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f88922-5de9-4a32-85d8-973519ff53c4 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:15.677607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e7b0-d472-4f23-8329-a3b992f27430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:15.679214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e7b0-d472-4f23-8329-a3b992f27430 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-11a9e7b0-d472-4f23-8329-a3b992f27430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:15.698441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:15.699349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:15.700216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:15.700216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:15.700216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:15.700216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:15.700216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 303/611] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:15.705420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad83bc-0a11-45b2-9397-af6759849470 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.705709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad83bc-0a11-45b2-9397-af6759849470 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.705987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad83bc-0a11-45b2-9397-af6759849470 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.707318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 304/612] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:15.708037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad83bc-0a11-45b2-9397-af6759849470 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:15.713453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a16d63-6665-4769-b074-269e94eebf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:15.714348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16d63-6665-4769-b074-269e94eebf20 None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.714639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16d63-6665-4769-b074-269e94eebf20 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.714918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16d63-6665-4769-b074-269e94eebf20 None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/e38a311f2b614a3598f6b51b3a235178/roles/147f73b08d294eae9859df32606b7d69` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.736391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:15.745535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a16d63-6665-4769-b074-269e94eebf20 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=e38a311f2b614a3598f6b51b3a235178, role_id=147f73b08d294eae9859df32606b7d69)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:15.747773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a16d63-6665-4769-b074-269e94eebf20 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:15.769248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 309/613] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:36:15 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/e38a311f2b614a3598f6b51b3a235178/roles/147f73b08d294eae9859df32606b7d69 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:36:15.771830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31c563-cbb1-4229-9c8c-0bc4ef4892c9 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.772646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31c563-cbb1-4229-9c8c-0bc4ef4892c9 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.773453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31c563-cbb1-4229-9c8c-0bc4ef4892c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.798145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-fc31c563-cbb1-4229-9c8c-0bc4ef4892c9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:15.800751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-fc31c563-cbb1-4229-9c8c-0bc4ef4892c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:15.837476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad83bc-0a11-45b2-9397-af6759849470 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:15.892895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 310/614] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:15.912611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d4783-f507-4b0c-8556-b80e4326c160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:15.913281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d4783-f507-4b0c-8556-b80e4326c160 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:15.913570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d4783-f507-4b0c-8556-b80e4326c160 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:15.913853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d4783-f507-4b0c-8556-b80e4326c160 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:15.914591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d4783-f507-4b0c-8556-b80e4326c160 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:15.914904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d4783-f507-4b0c-8556-b80e4326c160 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:15.916571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d4783-f507-4b0c-8556-b80e4326c160 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:15.922721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31c563-cbb1-4229-9c8c-0bc4ef4892c9 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:15.942619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 305/615] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:15.949525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d4783-f507-4b0c-8556-b80e4326c160 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:16.034275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:16.056457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31c563-cbb1-4229-9c8c-0bc4ef4892c9 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:16.084369 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:16.103288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e9949-1d35-4b06-969a-0704d82bc322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:16.104027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e9949-1d35-4b06-969a-0704d82bc322 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:16.104315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e9949-1d35-4b06-969a-0704d82bc322 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:16.104606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e9949-1d35-4b06-969a-0704d82bc322 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:16.116063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e9949-1d35-4b06-969a-0704d82bc322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-a64e9949-1d35-4b06-969a-0704d82bc322 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:16.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:16.125683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:16.126535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:16.127153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 311/616] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:16.128983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d4783-f507-4b0c-8556-b80e4326c160 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:16.142196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e9949-1d35-4b06-969a-0704d82bc322 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:16.181423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf01590-54b0-46eb-8864-56a42c0f399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:16.182351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf01590-54b0-46eb-8864-56a42c0f399a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:16.182809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf01590-54b0-46eb-8864-56a42c0f399a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:16.183131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf01590-54b0-46eb-8864-56a42c0f399a None admin] PATH_INFO: `/v3/users/swift` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:16.200529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf01590-54b0-46eb-8864-56a42c0f399a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:16.202176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf01590-54b0-46eb-8864-56a42c0f399a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-ccf01590-54b0-46eb-8864-56a42c0f399a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:16.234400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:16.235616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:16.236534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:16.237108 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 306/617] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:16.245219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf01590-54b0-46eb-8864-56a42c0f399a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:16.276237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e9949-1d35-4b06-969a-0704d82bc322 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:16.323766 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9d8eb-8607-4abd-9539-25a353af4da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:16.324764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9d8eb-8607-4abd-9539-25a353af4da4 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:16.325227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9d8eb-8607-4abd-9539-25a353af4da4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:16.325702 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9d8eb-8607-4abd-9539-25a353af4da4 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:16.326658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9d8eb-8607-4abd-9539-25a353af4da4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:16.327144 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9d8eb-8607-4abd-9539-25a353af4da4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:16.331206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9d8eb-8607-4abd-9539-25a353af4da4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:16.355157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 312/618] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:16.377606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9d8eb-8607-4abd-9539-25a353af4da4 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:16.402894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:16.410019 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf01590-54b0-46eb-8864-56a42c0f399a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:16.460965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dd2c1-f09e-4451-a5ac-2a545174059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:16.461655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dd2c1-f09e-4451-a5ac-2a545174059d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:16.461936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dd2c1-f09e-4451-a5ac-2a545174059d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:16.462212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770dd2c1-f09e-4451-a5ac-2a545174059d None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:16.463062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dd2c1-f09e-4451-a5ac-2a545174059d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:16.463370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dd2c1-f09e-4451-a5ac-2a545174059d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:16.464826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770dd2c1-f09e-4451-a5ac-2a545174059d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:16.465288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:16.504237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 307/619] 10.222.0.47 () {56 vars in 1135 bytes} [Fri Jun 24 23:36:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:16.509369 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dd2c1-f09e-4451-a5ac-2a545174059d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:16.530057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9d8eb-8607-4abd-9539-25a353af4da4 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:16.567151 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d205e66-98d9-4dd8-9fd2-8ba1b705998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:16.568185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d205e66-98d9-4dd8-9fd2-8ba1b705998c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:16.568677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d205e66-98d9-4dd8-9fd2-8ba1b705998c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:16.569100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d205e66-98d9-4dd8-9fd2-8ba1b705998c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:16.581507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d205e66-98d9-4dd8-9fd2-8ba1b705998c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:16.583152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d205e66-98d9-4dd8-9fd2-8ba1b705998c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-3d205e66-98d9-4dd8-9fd2-8ba1b705998c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:16.601643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:16.602899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:16.603987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:16.603987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:16.603987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:16.603987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:16.605917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 313/620] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:16.615657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d205e66-98d9-4dd8-9fd2-8ba1b705998c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:16.744635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770dd2c1-f09e-4451-a5ac-2a545174059d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:16.779746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:16.794127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d205e66-98d9-4dd8-9fd2-8ba1b705998c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:16.805340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07553ba-bfce-40ff-b5d4-82760d4ac776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:16.806161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07553ba-bfce-40ff-b5d4-82760d4ac776 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:16.806455 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07553ba-bfce-40ff-b5d4-82760d4ac776 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:16.806743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07553ba-bfce-40ff-b5d4-82760d4ac776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:16.827075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07553ba-bfce-40ff-b5d4-82760d4ac776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:16.829266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07553ba-bfce-40ff-b5d4-82760d4ac776 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:16.848635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab9cf0-37f7-4127-8391-6d53428cb50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:16.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab9cf0-37f7-4127-8391-6d53428cb50b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:16.850148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab9cf0-37f7-4127-8391-6d53428cb50b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:16.850647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab9cf0-37f7-4127-8391-6d53428cb50b None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:16.851548 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab9cf0-37f7-4127-8391-6d53428cb50b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:16.852129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab9cf0-37f7-4127-8391-6d53428cb50b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:16.853698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab9cf0-37f7-4127-8391-6d53428cb50b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-d07553ba-bfce-40ff-b5d4-82760d4ac776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:16.860956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:16.861837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:16.862956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:16.862956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:16.862956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:16.862956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:16.862956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 308/621] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:16.869752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07553ba-bfce-40ff-b5d4-82760d4ac776 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:16.870624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 314/622] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:16.880921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:16.884812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab9cf0-37f7-4127-8391-6d53428cb50b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:17.018792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07553ba-bfce-40ff-b5d4-82760d4ac776 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:17.026172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab9cf0-37f7-4127-8391-6d53428cb50b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:17.075289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab645d-a640-4da2-bb50-410adf21f052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:17.076102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab645d-a640-4da2-bb50-410adf21f052 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.076385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab645d-a640-4da2-bb50-410adf21f052 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.076684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab645d-a640-4da2-bb50-410adf21f052 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.079057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6009a-0fd5-4433-9f4a-c1c12ff15fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:17.079713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6009a-0fd5-4433-9f4a-c1c12ff15fc5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.079993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6009a-0fd5-4433-9f4a-c1c12ff15fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.080268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6009a-0fd5-4433-9f4a-c1c12ff15fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.081035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6009a-0fd5-4433-9f4a-c1c12ff15fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:17.081318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6009a-0fd5-4433-9f4a-c1c12ff15fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:17.082726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6009a-0fd5-4433-9f4a-c1c12ff15fc5 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:17.097050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab645d-a640-4da2-bb50-410adf21f052 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:17.098510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab645d-a640-4da2-bb50-410adf21f052 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:17.111482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 309/623] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:17.122004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6009a-0fd5-4433-9f4a-c1c12ff15fc5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-87ab645d-a640-4da2-bb50-410adf21f052 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:17.126543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:17.128218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 23:36:17.129448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:17.143735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 315/624] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:17.162406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab645d-a640-4da2-bb50-410adf21f052 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:17.188352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:17.228396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:17.278952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6009a-0fd5-4433-9f4a-c1c12ff15fc5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:17.319923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab645d-a640-4da2-bb50-410adf21f052 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:17.324233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258be88f-d883-45d0-abc8-6cd42014576f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:17.325071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258be88f-d883-45d0-abc8-6cd42014576f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.325358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258be88f-d883-45d0-abc8-6cd42014576f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.325640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258be88f-d883-45d0-abc8-6cd42014576f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.335808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258be88f-d883-45d0-abc8-6cd42014576f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:17.337765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258be88f-d883-45d0-abc8-6cd42014576f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:17.356740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb665a-2d92-483a-8199-65d72ea0a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-258be88f-d883-45d0-abc8-6cd42014576f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:17.357158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:17.358017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:17.360443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:17.360443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:17.360443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:17.360443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:17.360443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:17.360443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:17.360443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:17.360443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:17.360896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb665a-2d92-483a-8199-65d72ea0a199 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.360896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb665a-2d92-483a-8199-65d72ea0a199 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.360896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb665a-2d92-483a-8199-65d72ea0a199 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.361084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 310/625] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:17.361151 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb665a-2d92-483a-8199-65d72ea0a199 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:17.361151 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb665a-2d92-483a-8199-65d72ea0a199 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:17.361301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd473a-0e48-454b-8c26-54f95f05e00b None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.361301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd473a-0e48-454b-8c26-54f95f05e00b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.361301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd473a-0e48-454b-8c26-54f95f05e00b None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.361538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb665a-2d92-483a-8199-65d72ea0a199 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:17.366635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 311/626] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:17.367794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e90af-2470-4159-9e31-ae385af769ee None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.368083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e90af-2470-4159-9e31-ae385af769ee None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.368360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e90af-2470-4159-9e31-ae385af769ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.385122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7d2e90af-2470-4159-9e31-ae385af769ee None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:17.386367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7d2e90af-2470-4159-9e31-ae385af769ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:17.388393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 316/627] 10.222.0.47 () {56 vars in 1137 bytes} [Fri Jun 24 23:36:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:17.389466 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb665a-2d92-483a-8199-65d72ea0a199 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:17.476605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 312/628] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:36:17.476605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e90af-2470-4159-9e31-ae385af769ee None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:17.503496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:17.536222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb665a-2d92-483a-8199-65d72ea0a199 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:17.551943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:17.573983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b954b-367f-49e7-b311-fcee069beba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:17.574653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b954b-367f-49e7-b311-fcee069beba2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.574938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b954b-367f-49e7-b311-fcee069beba2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.575216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b954b-367f-49e7-b311-fcee069beba2 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.576008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b954b-367f-49e7-b311-fcee069beba2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:17.576315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b954b-367f-49e7-b311-fcee069beba2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:17.578668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b954b-367f-49e7-b311-fcee069beba2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:17.596734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 317/629] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:17.597835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b954b-367f-49e7-b311-fcee069beba2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:17.609635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e90af-2470-4159-9e31-ae385af769ee None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:17.648042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3eb385-048d-4ac3-9db4-d2af4ccdc359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:17.649193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3eb385-048d-4ac3-9db4-d2af4ccdc359 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.649828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3eb385-048d-4ac3-9db4-d2af4ccdc359 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.650318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3eb385-048d-4ac3-9db4-d2af4ccdc359 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.701358 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b954b-367f-49e7-b311-fcee069beba2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:17.723118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3eb385-048d-4ac3-9db4-d2af4ccdc359 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:17.724597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3eb385-048d-4ac3-9db4-d2af4ccdc359 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:17.732732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:17.738578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949bb1e-82f0-4105-8387-4c9c24c2b162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:17.739995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949bb1e-82f0-4105-8387-4c9c24c2b162 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.740232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949bb1e-82f0-4105-8387-4c9c24c2b162 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.740512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949bb1e-82f0-4105-8387-4c9c24c2b162 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.752450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949bb1e-82f0-4105-8387-4c9c24c2b162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-3949bb1e-82f0-4105-8387-4c9c24c2b162 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:17.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:17.761042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:17.761866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 318/630] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:17.762552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55f0ed-a16c-4b2d-a3a9-44792d2046fd None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.763236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55f0ed-a16c-4b2d-a3a9-44792d2046fd None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.768087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55f0ed-a16c-4b2d-a3a9-44792d2046fd None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.769708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 319/631] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:17.774001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55f0ed-a16c-4b2d-a3a9-44792d2046fd None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:17.799802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:17.851537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 313/632] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:17.852270 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3eb385-048d-4ac3-9db4-d2af4ccdc359 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:17.891034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55f0ed-a16c-4b2d-a3a9-44792d2046fd None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:17.932428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397dad6-35e7-4468-a73f-709be6dfbb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:17.933305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397dad6-35e7-4468-a73f-709be6dfbb9f None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.933930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397dad6-35e7-4468-a73f-709be6dfbb9f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.934228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397dad6-35e7-4468-a73f-709be6dfbb9f None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/3dd1e02af2374224b968c2644dd0c813/roles/147f73b08d294eae9859df32606b7d69` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.950886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397dad6-35e7-4468-a73f-709be6dfbb9f None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=3dd1e02af2374224b968c2644dd0c813, role_id=147f73b08d294eae9859df32606b7d69)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:17.952333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397dad6-35e7-4468-a73f-709be6dfbb9f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:17.957968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3eb385-048d-4ac3-9db4-d2af4ccdc359 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:17.970358 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 320/633] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:36:17 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/3dd1e02af2374224b968c2644dd0c813/roles/147f73b08d294eae9859df32606b7d69 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:36:17.971684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c7120-5689-46c8-8ecc-d2abcd88156a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:17.971981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c7120-5689-46c8-8ecc-d2abcd88156a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:17.972263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c7120-5689-46c8-8ecc-d2abcd88156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:17.997435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-3d8c7120-5689-46c8-8ecc-d2abcd88156a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:17.998668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3d8c7120-5689-46c8-8ecc-d2abcd88156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:18.010724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fcf04-99b5-4024-97f8-8b7f92da7b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.012123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fcf04-99b5-4024-97f8-8b7f92da7b95 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.012773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fcf04-99b5-4024-97f8-8b7f92da7b95 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.013337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fcf04-99b5-4024-97f8-8b7f92da7b95 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.014301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fcf04-99b5-4024-97f8-8b7f92da7b95 None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:18.015496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fcf04-99b5-4024-97f8-8b7f92da7b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:18.017948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fcf04-99b5-4024-97f8-8b7f92da7b95 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:18.029599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 314/634] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:18.031447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fcf04-99b5-4024-97f8-8b7f92da7b95 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.047818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:18.061607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:18.090089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 321/635] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:18.096654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c7120-5689-46c8-8ecc-d2abcd88156a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.170348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fcf04-99b5-4024-97f8-8b7f92da7b95 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:18.208895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e1456-ae79-4e6c-8012-636d1e00136a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.209717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e1456-ae79-4e6c-8012-636d1e00136a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.209897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e1456-ae79-4e6c-8012-636d1e00136a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.210169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e1456-ae79-4e6c-8012-636d1e00136a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.221674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e1456-ae79-4e6c-8012-636d1e00136a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:18.223647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e1456-ae79-4e6c-8012-636d1e00136a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:18.228370 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c7120-5689-46c8-8ecc-d2abcd88156a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-c92e1456-ae79-4e6c-8012-636d1e00136a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:18.241753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:18.242662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:18.243507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:18.243507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:18.243507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:18.243507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:18.243507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 315/636] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:18.243767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e1456-ae79-4e6c-8012-636d1e00136a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.270413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f112316-9291-4466-8ef7-64f763a059ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.271214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f112316-9291-4466-8ef7-64f763a059ce None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.271502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f112316-9291-4466-8ef7-64f763a059ce None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.271781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f112316-9291-4466-8ef7-64f763a059ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.284124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f112316-9291-4466-8ef7-64f763a059ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:18.285601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f112316-9291-4466-8ef7-64f763a059ce None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-7f112316-9291-4466-8ef7-64f763a059ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:18.304412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:18.305371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:18.306211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:18.306211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:18.306211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:18.306211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:18.306211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 322/637] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:18.307016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dbbd8-cf55-4c5d-b2e4-7a631ce7d9be None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.307306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dbbd8-cf55-4c5d-b2e4-7a631ce7d9be None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.307586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dbbd8-cf55-4c5d-b2e4-7a631ce7d9be None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.312622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 323/638] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:18.312622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dbbd8-cf55-4c5d-b2e4-7a631ce7d9be None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.325042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:18.339810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:18.362250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e1456-ae79-4e6c-8012-636d1e00136a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:18.400241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814f56f-ee06-4e5f-9dfb-8b46405a4cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.401277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814f56f-ee06-4e5f-9dfb-8b46405a4cc9 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.401416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814f56f-ee06-4e5f-9dfb-8b46405a4cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.401598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814f56f-ee06-4e5f-9dfb-8b46405a4cc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.413371 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814f56f-ee06-4e5f-9dfb-8b46405a4cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-f814f56f-ee06-4e5f-9dfb-8b46405a4cc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:18.421169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:18.422684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:18.428633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:18.429307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dbbd8-cf55-4c5d-b2e4-7a631ce7d9be None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:18.429387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 316/639] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:18.429387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814f56f-ee06-4e5f-9dfb-8b46405a4cc9 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.470421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfd705-b622-4ee0-b7f3-e847dd46c01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.471399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd705-b622-4ee0-b7f3-e847dd46c01b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.471892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd705-b622-4ee0-b7f3-e847dd46c01b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.472177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfd705-b622-4ee0-b7f3-e847dd46c01b None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.472959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfd705-b622-4ee0-b7f3-e847dd46c01b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:18.473274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfd705-b622-4ee0-b7f3-e847dd46c01b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:18.474649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfd705-b622-4ee0-b7f3-e847dd46c01b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:18.487491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 324/640] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:18.489910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c338a-9a69-46a0-acf9-30d453ded669 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.490202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c338a-9a69-46a0-acf9-30d453ded669 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.490480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c338a-9a69-46a0-acf9-30d453ded669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.507809 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0d8c338a-9a69-46a0-acf9-30d453ded669 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:18.509115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0d8c338a-9a69-46a0-acf9-30d453ded669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:18.539827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814f56f-ee06-4e5f-9dfb-8b46405a4cc9 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:18.572708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:18.584686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 325/641] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:18.584686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c338a-9a69-46a0-acf9-30d453ded669 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.588660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece29aa5-d3b2-44e3-9f7c-861957a22556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.589549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece29aa5-d3b2-44e3-9f7c-861957a22556 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.589909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece29aa5-d3b2-44e3-9f7c-861957a22556 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.590262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece29aa5-d3b2-44e3-9f7c-861957a22556 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.591143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece29aa5-d3b2-44e3-9f7c-861957a22556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:18.591526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece29aa5-d3b2-44e3-9f7c-861957a22556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:18.593011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:18.594552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece29aa5-d3b2-44e3-9f7c-861957a22556 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:18.608111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 317/642] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:18.609192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece29aa5-d3b2-44e3-9f7c-861957a22556 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.688546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c338a-9a69-46a0-acf9-30d453ded669 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:18.713967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece29aa5-d3b2-44e3-9f7c-861957a22556 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:18.724443 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce204fd1-f1ce-46c4-8c34-02733b057c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.725079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce204fd1-f1ce-46c4-8c34-02733b057c12 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.725362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce204fd1-f1ce-46c4-8c34-02733b057c12 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.725641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce204fd1-f1ce-46c4-8c34-02733b057c12 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.726387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce204fd1-f1ce-46c4-8c34-02733b057c12 None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:18.726694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce204fd1-f1ce-46c4-8c34-02733b057c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:18.728043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce204fd1-f1ce-46c4-8c34-02733b057c12 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:18.736330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 326/643] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:18.740913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce204fd1-f1ce-46c4-8c34-02733b057c12 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.755981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9f850-66ea-4c7f-8bc9-48eae12317c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.757062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9f850-66ea-4c7f-8bc9-48eae12317c3 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.757526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9f850-66ea-4c7f-8bc9-48eae12317c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.757953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9f850-66ea-4c7f-8bc9-48eae12317c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.765851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9f850-66ea-4c7f-8bc9-48eae12317c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:18.767544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9f850-66ea-4c7f-8bc9-48eae12317c3 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-32f9f850-66ea-4c7f-8bc9-48eae12317c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:18.781819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:18.782918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:18.784013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:18.784013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:18.784013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:18.784013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:18.784013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:18.784013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:18.784013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:18.784013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:18.785773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 318/644] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:18.786709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9f850-66ea-4c7f-8bc9-48eae12317c3 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.838796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:18.848305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:18.867506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce204fd1-f1ce-46c4-8c34-02733b057c12 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:18.903372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9f850-66ea-4c7f-8bc9-48eae12317c3 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:18.904839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa753d-062b-4c4d-94b6-c49181080e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.905709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa753d-062b-4c4d-94b6-c49181080e63 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.906563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa753d-062b-4c4d-94b6-c49181080e63 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.906563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa753d-062b-4c4d-94b6-c49181080e63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.917112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa753d-062b-4c4d-94b6-c49181080e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-3dfa753d-062b-4c4d-94b6-c49181080e63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:18.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:18.926385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:18.927249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 327/645] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:18.928032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa753d-062b-4c4d-94b6-c49181080e63 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:18.948461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f49c9-344e-40c4-beb3-a9c7f7fcd336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:18.949327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f49c9-344e-40c4-beb3-a9c7f7fcd336 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:18.949617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f49c9-344e-40c4-beb3-a9c7f7fcd336 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:18.949899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f49c9-344e-40c4-beb3-a9c7f7fcd336 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:18.961222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f49c9-344e-40c4-beb3-a9c7f7fcd336 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:18.962690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f49c9-344e-40c4-beb3-a9c7f7fcd336 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-f53f49c9-344e-40c4-beb3-a9c7f7fcd336 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:18.982201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:18.983128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:18.984694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:18.986126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 319/646] 10.222.0.47 () {56 vars in 1093 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:18.987028 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f49c9-344e-40c4-beb3-a9c7f7fcd336 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.042081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa753d-062b-4c4d-94b6-c49181080e63 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:19.083709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29b68a-eb26-47a0-bc3a-ab6fcebc8629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:19.084512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29b68a-eb26-47a0-bc3a-ab6fcebc8629 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:19.084817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29b68a-eb26-47a0-bc3a-ab6fcebc8629 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:19.085088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29b68a-eb26-47a0-bc3a-ab6fcebc8629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:19.088134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:19.097304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f49c9-344e-40c4-beb3-a9c7f7fcd336 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:19.098393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29b68a-eb26-47a0-bc3a-ab6fcebc8629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:19.099900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29b68a-eb26-47a0-bc3a-ab6fcebc8629 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:19.109165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-db29b68a-eb26-47a0-bc3a-ab6fcebc8629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:19.117978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:19.118956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:19.119822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:19.119822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:19.119822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:19.119822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:19.124124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 328/647] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:19.125874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29b68a-eb26-47a0-bc3a-ab6fcebc8629 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.138378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09d7bc-afd7-4bdf-a425-f9c6b8272440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:19.139376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09d7bc-afd7-4bdf-a425-f9c6b8272440 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:19.139739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09d7bc-afd7-4bdf-a425-f9c6b8272440 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:19.140028 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09d7bc-afd7-4bdf-a425-f9c6b8272440 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:19.141359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09d7bc-afd7-4bdf-a425-f9c6b8272440 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:19.141842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09d7bc-afd7-4bdf-a425-f9c6b8272440 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:19.143514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09d7bc-afd7-4bdf-a425-f9c6b8272440 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:19.157009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 320/648] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:19.157723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09d7bc-afd7-4bdf-a425-f9c6b8272440 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.247390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29b68a-eb26-47a0-bc3a-ab6fcebc8629 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:19.260881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09d7bc-afd7-4bdf-a425-f9c6b8272440 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:19.284629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed38c6-af1e-4474-af61-78b28b358fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:19.284972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed38c6-af1e-4474-af61-78b28b358fa8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:19.285251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed38c6-af1e-4474-af61-78b28b358fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:19.285533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed38c6-af1e-4474-af61-78b28b358fa8 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:19.286277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed38c6-af1e-4474-af61-78b28b358fa8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:19.286585 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed38c6-af1e-4474-af61-78b28b358fa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:19.287942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed38c6-af1e-4474-af61-78b28b358fa8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:19.296009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 329/649] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:19.297576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed38c6-af1e-4474-af61-78b28b358fa8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.305641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629951e2-8a9d-4602-b7c3-8818225fc2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:19.306997 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629951e2-8a9d-4602-b7c3-8818225fc2a6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:19.307458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629951e2-8a9d-4602-b7c3-8818225fc2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:19.308339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629951e2-8a9d-4602-b7c3-8818225fc2a6 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:19.309772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629951e2-8a9d-4602-b7c3-8818225fc2a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:19.310241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629951e2-8a9d-4602-b7c3-8818225fc2a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:19.311983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629951e2-8a9d-4602-b7c3-8818225fc2a6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:19.329849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 321/650] 10.222.0.47 () {56 vars in 1133 bytes} [Fri Jun 24 23:36:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:19.330681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629951e2-8a9d-4602-b7c3-8818225fc2a6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.352999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:19.362934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:19.406942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed38c6-af1e-4474-af61-78b28b358fa8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:19.438047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629951e2-8a9d-4602-b7c3-8818225fc2a6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:19.443532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2889a40-b755-46e8-8264-746fdf00474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:19.444664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2889a40-b755-46e8-8264-746fdf00474e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:19.445112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2889a40-b755-46e8-8264-746fdf00474e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:19.445513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2889a40-b755-46e8-8264-746fdf00474e None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:19.446147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2889a40-b755-46e8-8264-746fdf00474e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:19.446464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2889a40-b755-46e8-8264-746fdf00474e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:19.447857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2889a40-b755-46e8-8264-746fdf00474e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:19.461696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 330/651] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:19.462850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2889a40-b755-46e8-8264-746fdf00474e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.474767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decb11c-90f7-4b5c-83fe-fabafbd0c73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:19.475436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decb11c-90f7-4b5c-83fe-fabafbd0c73c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:19.475721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decb11c-90f7-4b5c-83fe-fabafbd0c73c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:19.475998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decb11c-90f7-4b5c-83fe-fabafbd0c73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:19.476874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decb11c-90f7-4b5c-83fe-fabafbd0c73c None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=71e5ef9dc32f494791ba4addb5151232) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:19.477190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decb11c-90f7-4b5c-83fe-fabafbd0c73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:19.478629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decb11c-90f7-4b5c-83fe-fabafbd0c73c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:19.485476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 322/652] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:19 2022] GET /identity/v3/role_assignments?user.id=71e5ef9dc32f494791ba4addb5151232&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:19.486122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decb11c-90f7-4b5c-83fe-fabafbd0c73c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.592390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:19.604312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decb11c-90f7-4b5c-83fe-fabafbd0c73c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:19.619584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2889a40-b755-46e8-8264-746fdf00474e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:19.633898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:19.662583 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f3f36-1282-4006-a138-62b565e5d885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:19.663252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b00b5-aaf4-4cf4-a2d2-1cf800569ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:19.663692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f3f36-1282-4006-a138-62b565e5d885 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:19.664130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f3f36-1282-4006-a138-62b565e5d885 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:19.664340 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b00b5-aaf4-4cf4-a2d2-1cf800569ca0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:19.664736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f3f36-1282-4006-a138-62b565e5d885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:19.664857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b00b5-aaf4-4cf4-a2d2-1cf800569ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:19.665208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b00b5-aaf4-4cf4-a2d2-1cf800569ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:19.678733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b00b5-aaf4-4cf4-a2d2-1cf800569ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:19.680197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b00b5-aaf4-4cf4-a2d2-1cf800569ca0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-e68b00b5-aaf4-4cf4-a2d2-1cf800569ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:19.703936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:19.704927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:19.705845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:19.705845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:19.705845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:19.705845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:19.706063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 331/653] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:19.707744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b00b5-aaf4-4cf4-a2d2-1cf800569ca0 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.767050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f3f36-1282-4006-a138-62b565e5d885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:19.768685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0f3f36-1282-4006-a138-62b565e5d885 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:19.823589 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b00b5-aaf4-4cf4-a2d2-1cf800569ca0 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:19.875301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19480b93-c626-4ba7-9951-135885c86d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:19.876933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19480b93-c626-4ba7-9951-135885c86d6e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:19.877365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19480b93-c626-4ba7-9951-135885c86d6e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:19.877807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19480b93-c626-4ba7-9951-135885c86d6e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:19.888448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19480b93-c626-4ba7-9951-135885c86d6e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:19.890707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19480b93-c626-4ba7-9951-135885c86d6e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:19.894301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:19.910595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 323/654] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:19.911613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f3f36-1282-4006-a138-62b565e5d885 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-19480b93-c626-4ba7-9951-135885c86d6e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:19.911975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:19.912999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 23:36:19.913894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:19.917391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 332/655] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:19.918506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19480b93-c626-4ba7-9951-135885c86d6e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:19.957864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:20.029389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f3f36-1282-4006-a138-62b565e5d885 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.066233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19480b93-c626-4ba7-9951-135885c86d6e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.067440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4d84e-b182-4df6-b8ee-b561a2db47d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.068340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4d84e-b182-4df6-b8ee-b561a2db47d6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.068759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4d84e-b182-4df6-b8ee-b561a2db47d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.069232 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4d84e-b182-4df6-b8ee-b561a2db47d6 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.070467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4d84e-b182-4df6-b8ee-b561a2db47d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:20.070585 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4d84e-b182-4df6-b8ee-b561a2db47d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:20.072039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4d84e-b182-4df6-b8ee-b561a2db47d6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:20.086300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 324/656] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:20.087221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4d84e-b182-4df6-b8ee-b561a2db47d6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:20.109886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8680d2-882f-4faa-8637-7cef006a61c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.110687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8680d2-882f-4faa-8637-7cef006a61c8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.110968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8680d2-882f-4faa-8637-7cef006a61c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.111247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8680d2-882f-4faa-8637-7cef006a61c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.121432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8680d2-882f-4faa-8637-7cef006a61c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:20.122866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8680d2-882f-4faa-8637-7cef006a61c8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-5c8680d2-882f-4faa-8637-7cef006a61c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:20.141907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:20.142792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:20.143679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:20.143679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:20.143679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:20.143679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:20.143679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 333/657] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:20.144831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b598c8-d662-4671-b159-2a2b66d9c9fa None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.144831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b598c8-d662-4671-b159-2a2b66d9c9fa None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.144831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b598c8-d662-4671-b159-2a2b66d9c9fa None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.145562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:20.146408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 334/658] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:20.147125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b598c8-d662-4671-b159-2a2b66d9c9fa None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:20.211159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4d84e-b182-4df6-b8ee-b561a2db47d6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.228334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:20.244561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bb82-33c9-412b-83e0-dbc734eab5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.245257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bb82-33c9-412b-83e0-dbc734eab5b1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.245543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bb82-33c9-412b-83e0-dbc734eab5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.245813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bb82-33c9-412b-83e0-dbc734eab5b1 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.246654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bb82-33c9-412b-83e0-dbc734eab5b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:20.246964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bb82-33c9-412b-83e0-dbc734eab5b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:20.248395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bb82-33c9-412b-83e0-dbc734eab5b1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:20.261684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b598c8-d662-4671-b159-2a2b66d9c9fa None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.267439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 325/659] 10.222.0.47 () {56 vars in 1137 bytes} [Fri Jun 24 23:36:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:20.268278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bb82-33c9-412b-83e0-dbc734eab5b1 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:20.304096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ebcc8-0aa3-4381-aa21-b44f61506316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.305181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ebcc8-0aa3-4381-aa21-b44f61506316 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.305181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ebcc8-0aa3-4381-aa21-b44f61506316 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.305464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ebcc8-0aa3-4381-aa21-b44f61506316 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.313229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ebcc8-0aa3-4381-aa21-b44f61506316 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:20.314677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ebcc8-0aa3-4381-aa21-b44f61506316 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-cb4ebcc8-0aa3-4381-aa21-b44f61506316 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:20.330016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:20.330943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:20.331853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 335/660] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:20.332449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d214b-e742-4f06-b837-213633a6dd0e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.333054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d214b-e742-4f06-b837-213633a6dd0e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.333054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d214b-e742-4f06-b837-213633a6dd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.349402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f07d214b-e742-4f06-b837-213633a6dd0e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:20.350585 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f07d214b-e742-4f06-b837-213633a6dd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:20.382048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bb82-33c9-412b-83e0-dbc734eab5b1 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.393837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:20.420392 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c86945-f8a0-4a21-91f2-a1e2ee9f62c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.421836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c86945-f8a0-4a21-91f2-a1e2ee9f62c5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.422129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c86945-f8a0-4a21-91f2-a1e2ee9f62c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.422411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c86945-f8a0-4a21-91f2-a1e2ee9f62c5 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.423133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c86945-f8a0-4a21-91f2-a1e2ee9f62c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:20.423445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c86945-f8a0-4a21-91f2-a1e2ee9f62c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:20.424849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c86945-f8a0-4a21-91f2-a1e2ee9f62c5 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:20.427691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 336/661] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:20.437779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d214b-e742-4f06-b837-213633a6dd0e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:20.447133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 326/662] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:20.448018 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c86945-f8a0-4a21-91f2-a1e2ee9f62c5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:20.500209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:20.541147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d214b-e742-4f06-b837-213633a6dd0e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.554153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c86945-f8a0-4a21-91f2-a1e2ee9f62c5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.575341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06430254-f828-4255-9282-fe81d48011ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.575994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06430254-f828-4255-9282-fe81d48011ae None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.576284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06430254-f828-4255-9282-fe81d48011ae None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.576567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06430254-f828-4255-9282-fe81d48011ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.594966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42fd89-a769-4021-9606-5d6404a50b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.595646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42fd89-a769-4021-9606-5d6404a50b6f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.595951 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42fd89-a769-4021-9606-5d6404a50b6f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.596210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42fd89-a769-4021-9606-5d6404a50b6f None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.597089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42fd89-a769-4021-9606-5d6404a50b6f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:20.597400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42fd89-a769-4021-9606-5d6404a50b6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:20.598830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42fd89-a769-4021-9606-5d6404a50b6f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:20.621238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 327/663] 10.222.0.47 () {56 vars in 1139 bytes} [Fri Jun 24 23:36:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:20.621238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42fd89-a769-4021-9606-5d6404a50b6f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:20.649733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:20.663710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06430254-f828-4255-9282-fe81d48011ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:20.665657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06430254-f828-4255-9282-fe81d48011ae None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:20.725244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42fd89-a769-4021-9606-5d6404a50b6f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.742501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:20.764128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35584cba-dcbc-4fa0-bbf0-3789b59a9d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.765136 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584cba-dcbc-4fa0-bbf0-3789b59a9d0f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.765512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584cba-dcbc-4fa0-bbf0-3789b59a9d0f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.766039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35584cba-dcbc-4fa0-bbf0-3789b59a9d0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.774307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35584cba-dcbc-4fa0-bbf0-3789b59a9d0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:20.776321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35584cba-dcbc-4fa0-bbf0-3789b59a9d0f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:20.790339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 337/664] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:20.790339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06430254-f828-4255-9282-fe81d48011ae None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-35584cba-dcbc-4fa0-bbf0-3789b59a9d0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:20.791414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:20.792245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:20.793156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:20.793156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:20.793156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:20.793156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:20.793156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:20.793156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:20.793156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:20.793156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:20.796728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 328/665] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:20.797814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35584cba-dcbc-4fa0-bbf0-3789b59a9d0f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:20.896800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:20.904571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06430254-f828-4255-9282-fe81d48011ae None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.911001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35584cba-dcbc-4fa0-bbf0-3789b59a9d0f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:20.943149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4074818-a79b-41d1-b4b6-1a947cda17c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.943956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4074818-a79b-41d1-b4b6-1a947cda17c7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.944257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4074818-a79b-41d1-b4b6-1a947cda17c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.944517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4074818-a79b-41d1-b4b6-1a947cda17c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.948356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e084e5a-7df7-494a-80c6-94ccfe54cc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:20.949297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e084e5a-7df7-494a-80c6-94ccfe54cc38 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:20.949649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e084e5a-7df7-494a-80c6-94ccfe54cc38 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:20.949926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e084e5a-7df7-494a-80c6-94ccfe54cc38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:20.961813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4074818-a79b-41d1-b4b6-1a947cda17c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-e4074818-a79b-41d1-b4b6-1a947cda17c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:20.971222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:20.972113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 338/666] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:20.973984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4074818-a79b-41d1-b4b6-1a947cda17c7 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.001028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:21.026907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e084e5a-7df7-494a-80c6-94ccfe54cc38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.028455 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e084e5a-7df7-494a-80c6-94ccfe54cc38 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.082263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4074818-a79b-41d1-b4b6-1a947cda17c7 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:21.119327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896713c-3344-414f-8c4f-2ca976bb23cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:21.120337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896713c-3344-414f-8c4f-2ca976bb23cf None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.120496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896713c-3344-414f-8c4f-2ca976bb23cf None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.120790 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896713c-3344-414f-8c4f-2ca976bb23cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.121166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:21.132299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896713c-3344-414f-8c4f-2ca976bb23cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.133786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896713c-3344-414f-8c4f-2ca976bb23cf None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.139806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 329/667] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:21.139806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e084e5a-7df7-494a-80c6-94ccfe54cc38 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-d896713c-3344-414f-8c4f-2ca976bb23cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:21.158907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:21.159794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:21.161120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:21.161120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:21.161120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:21.161120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:21.161120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 339/668] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:21.161120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896713c-3344-414f-8c4f-2ca976bb23cf None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.249832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:21.260484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e084e5a-7df7-494a-80c6-94ccfe54cc38 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:21.274603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896713c-3344-414f-8c4f-2ca976bb23cf None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:21.302552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b91c-d760-4d5e-8924-ecba80f3add1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:21.303220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b91c-d760-4d5e-8924-ecba80f3add1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.303502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b91c-d760-4d5e-8924-ecba80f3add1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.303775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b91c-d760-4d5e-8924-ecba80f3add1 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.304566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b91c-d760-4d5e-8924-ecba80f3add1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:21.304898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b91c-d760-4d5e-8924-ecba80f3add1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.306325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b91c-d760-4d5e-8924-ecba80f3add1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.319080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 330/669] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:21.319920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b91c-d760-4d5e-8924-ecba80f3add1 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.328817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaf18c-0e58-4176-95b7-b2bb092b3bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:21.329492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaf18c-0e58-4176-95b7-b2bb092b3bb1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.329869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaf18c-0e58-4176-95b7-b2bb092b3bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.330153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eaf18c-0e58-4176-95b7-b2bb092b3bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.330910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eaf18c-0e58-4176-95b7-b2bb092b3bb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:21.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eaf18c-0e58-4176-95b7-b2bb092b3bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.332625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eaf18c-0e58-4176-95b7-b2bb092b3bb1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.347314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 340/670] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:21.348004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaf18c-0e58-4176-95b7-b2bb092b3bb1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.375451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:21.435190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b91c-d760-4d5e-8924-ecba80f3add1 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:21.467154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eaf18c-0e58-4176-95b7-b2bb092b3bb1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:21.476178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82815191-e941-4fdc-a993-32aea45f3f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:21.477096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82815191-e941-4fdc-a993-32aea45f3f94 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.477381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82815191-e941-4fdc-a993-32aea45f3f94 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.477649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82815191-e941-4fdc-a993-32aea45f3f94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.489069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82815191-e941-4fdc-a993-32aea45f3f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.490537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82815191-e941-4fdc-a993-32aea45f3f94 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.506095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf117c-8acb-4c95-9735-3ab52a3c824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:21.507048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf117c-8acb-4c95-9735-3ab52a3c824c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.507471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf117c-8acb-4c95-9735-3ab52a3c824c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.507883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf117c-8acb-4c95-9735-3ab52a3c824c None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.508769 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf117c-8acb-4c95-9735-3ab52a3c824c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:21.509212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf117c-8acb-4c95-9735-3ab52a3c824c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-82815191-e941-4fdc-a993-32aea45f3f94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:21.509429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:21.510400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:21.511293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:21.511293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:21.511293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:21.511293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:21.511293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 331/671] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:21.511584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82815191-e941-4fdc-a993-32aea45f3f94 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.513047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf117c-8acb-4c95-9735-3ab52a3c824c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.523004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:21.535021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 341/672] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:21.535021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf117c-8acb-4c95-9735-3ab52a3c824c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.634521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:21.638698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82815191-e941-4fdc-a993-32aea45f3f94 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:21.649079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf117c-8acb-4c95-9735-3ab52a3c824c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:21.678996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7f66-50c9-48a1-aa8a-084d53f6c03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:21.680525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7f66-50c9-48a1-aa8a-084d53f6c03f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.681294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7f66-50c9-48a1-aa8a-084d53f6c03f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.681770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7f66-50c9-48a1-aa8a-084d53f6c03f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.683047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7f66-50c9-48a1-aa8a-084d53f6c03f None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=feb9bec9f1894e179e24a14b81b3f23d) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:21.683831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7f66-50c9-48a1-aa8a-084d53f6c03f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.687338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7f66-50c9-48a1-aa8a-084d53f6c03f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.689089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2696213-4789-414c-90e3-13d5e2894c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:21.689896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2696213-4789-414c-90e3-13d5e2894c6e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.690174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2696213-4789-414c-90e3-13d5e2894c6e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.690449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2696213-4789-414c-90e3-13d5e2894c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.704616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 332/673] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/role_assignments?user.id=feb9bec9f1894e179e24a14b81b3f23d&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:21.704616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7f66-50c9-48a1-aa8a-084d53f6c03f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.710131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2696213-4789-414c-90e3-13d5e2894c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.711711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2696213-4789-414c-90e3-13d5e2894c6e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-c2696213-4789-414c-90e3-13d5e2894c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:21.737012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:21.737978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:21.738852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:21.738852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:21.738852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:21.738852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:21.738852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 342/674] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:21.739269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2696213-4789-414c-90e3-13d5e2894c6e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.806831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:21.851368 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7f66-50c9-48a1-aa8a-084d53f6c03f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:21.884883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2696213-4789-414c-90e3-13d5e2894c6e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:21.891474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82563132-5f2b-4c4e-98e6-64570155dedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:21.892159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82563132-5f2b-4c4e-98e6-64570155dedb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.892440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82563132-5f2b-4c4e-98e6-64570155dedb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.892745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82563132-5f2b-4c4e-98e6-64570155dedb None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.893480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82563132-5f2b-4c4e-98e6-64570155dedb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:21.893789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82563132-5f2b-4c4e-98e6-64570155dedb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.895180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82563132-5f2b-4c4e-98e6-64570155dedb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.909861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 333/675] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:21.911120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82563132-5f2b-4c4e-98e6-64570155dedb None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:21.930716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1837d7-70c5-4b12-8365-18054864dd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:21.931852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1837d7-70c5-4b12-8365-18054864dd64 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.933167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1837d7-70c5-4b12-8365-18054864dd64 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.933167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1837d7-70c5-4b12-8365-18054864dd64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.941801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1837d7-70c5-4b12-8365-18054864dd64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:21.943277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1837d7-70c5-4b12-8365-18054864dd64 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:21.947316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-9a1837d7-70c5-4b12-8365-18054864dd64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:21.961332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:21.962174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:21.963058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:21.963058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:21.963058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:21.963058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:21.963058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:21.963058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:21.963058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:21.963058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:21.964061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 343/676] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:21.965628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a02b3a-a377-4c0f-a0e6-4a3ed367455a None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:21.965916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a02b3a-a377-4c0f-a0e6-4a3ed367455a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:21.966194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a02b3a-a377-4c0f-a0e6-4a3ed367455a None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:21.967629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 344/677] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:21.975392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a02b3a-a377-4c0f-a0e6-4a3ed367455a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.051015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82563132-5f2b-4c4e-98e6-64570155dedb None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.087451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1da909-085d-4321-afb4-1df7abc2182c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.088117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1da909-085d-4321-afb4-1df7abc2182c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.088390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1da909-085d-4321-afb4-1df7abc2182c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.088727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1da909-085d-4321-afb4-1df7abc2182c None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.089918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1da909-085d-4321-afb4-1df7abc2182c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:22.090431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1da909-085d-4321-afb4-1df7abc2182c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.091887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1da909-085d-4321-afb4-1df7abc2182c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:22.110863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 334/678] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:22.110863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bc15c-ba22-495e-a55d-5e02cad7fd43 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.110863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bc15c-ba22-495e-a55d-5e02cad7fd43 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.110863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66bc15c-ba22-495e-a55d-5e02cad7fd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.118125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a02b3a-a377-4c0f-a0e6-4a3ed367455a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.127416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b66bc15c-ba22-495e-a55d-5e02cad7fd43 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:22.128657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b66bc15c-ba22-495e-a55d-5e02cad7fd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:22.131088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:22.161620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0217b-01ff-45bd-81f0-da2efb76eee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.162419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0217b-01ff-45bd-81f0-da2efb76eee4 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.162705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0217b-01ff-45bd-81f0-da2efb76eee4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.162983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0217b-01ff-45bd-81f0-da2efb76eee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.170280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0217b-01ff-45bd-81f0-da2efb76eee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.172535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0217b-01ff-45bd-81f0-da2efb76eee4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-44e0217b-01ff-45bd-81f0-da2efb76eee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:22.188343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:22.190061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 345/679] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:22.191685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0217b-01ff-45bd-81f0-da2efb76eee4 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.225162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 335/680] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:22.225162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bc15c-ba22-495e-a55d-5e02cad7fd43 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.246088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:22.309254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0217b-01ff-45bd-81f0-da2efb76eee4 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.339043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66bc15c-ba22-495e-a55d-5e02cad7fd43 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.346832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32136b-1075-4061-a2a5-a0b84982c024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.347750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32136b-1075-4061-a2a5-a0b84982c024 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.347888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32136b-1075-4061-a2a5-a0b84982c024 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.348046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32136b-1075-4061-a2a5-a0b84982c024 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.348873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32136b-1075-4061-a2a5-a0b84982c024 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:22.349189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32136b-1075-4061-a2a5-a0b84982c024 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.350739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e32136b-1075-4061-a2a5-a0b84982c024 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:22.363543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 346/681] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:22.364435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32136b-1075-4061-a2a5-a0b84982c024 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.381772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66544996-a230-4e58-94a6-f2a9fe39b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.382572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66544996-a230-4e58-94a6-f2a9fe39b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.382855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66544996-a230-4e58-94a6-f2a9fe39b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.383129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66544996-a230-4e58-94a6-f2a9fe39b8b5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.390930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66544996-a230-4e58-94a6-f2a9fe39b8b5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.392440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66544996-a230-4e58-94a6-f2a9fe39b8b5 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:22.399185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-66544996-a230-4e58-94a6-f2a9fe39b8b5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:22.408529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:22.409522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:22.410525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:22.414372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 336/682] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:22.415015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66544996-a230-4e58-94a6-f2a9fe39b8b5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.484055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32136b-1075-4061-a2a5-a0b84982c024 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.504471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:22.527181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65a6d2-fdb4-4226-ae58-c58d063cc0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.527968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65a6d2-fdb4-4226-ae58-c58d063cc0d9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.528256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65a6d2-fdb4-4226-ae58-c58d063cc0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.528535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65a6d2-fdb4-4226-ae58-c58d063cc0d9 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.529369 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65a6d2-fdb4-4226-ae58-c58d063cc0d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:22.529685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65a6d2-fdb4-4226-ae58-c58d063cc0d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.531353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66544996-a230-4e58-94a6-f2a9fe39b8b5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.531809 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65a6d2-fdb4-4226-ae58-c58d063cc0d9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:22.549015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 347/683] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:22.549015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65a6d2-fdb4-4226-ae58-c58d063cc0d9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.572701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6280d6-fff9-4719-9b68-5add936240ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.573579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6280d6-fff9-4719-9b68-5add936240ce None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.573936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6280d6-fff9-4719-9b68-5add936240ce None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.574275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6280d6-fff9-4719-9b68-5add936240ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.585125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6280d6-fff9-4719-9b68-5add936240ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.586572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6280d6-fff9-4719-9b68-5add936240ce None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-fd6280d6-fff9-4719-9b68-5add936240ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:22.604352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:22.605305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:22.606180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:22.606180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:22.606180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:22.606180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:22.606180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 337/684] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:22.606467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6280d6-fff9-4719-9b68-5add936240ce None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.649506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:22.662673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65a6d2-fdb4-4226-ae58-c58d063cc0d9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.704747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30b94e-0bb4-45b4-a16d-3e964b3145b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.705384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30b94e-0bb4-45b4-a16d-3e964b3145b8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.705665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30b94e-0bb4-45b4-a16d-3e964b3145b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.705945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30b94e-0bb4-45b4-a16d-3e964b3145b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.706773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30b94e-0bb4-45b4-a16d-3e964b3145b8 None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=fb004ddd208849f6a5316c19e9a916aa) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:22.707083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30b94e-0bb4-45b4-a16d-3e964b3145b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.708489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30b94e-0bb4-45b4-a16d-3e964b3145b8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:22.712436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6280d6-fff9-4719-9b68-5add936240ce None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.715299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 348/685] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:22 2022] GET /identity/v3/role_assignments?user.id=fb004ddd208849f6a5316c19e9a916aa&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:22.715733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30b94e-0bb4-45b4-a16d-3e964b3145b8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.756976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:22.769927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96b16b-8f71-4213-a0c1-e826f3b84479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.771582 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96b16b-8f71-4213-a0c1-e826f3b84479 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.772037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96b16b-8f71-4213-a0c1-e826f3b84479 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.772495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96b16b-8f71-4213-a0c1-e826f3b84479 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.774195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96b16b-8f71-4213-a0c1-e826f3b84479 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:22.774696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96b16b-8f71-4213-a0c1-e826f3b84479 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.776710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96b16b-8f71-4213-a0c1-e826f3b84479 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:22.798289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 338/686] 10.222.0.47 () {56 vars in 1135 bytes} [Fri Jun 24 23:36:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:22.799373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96b16b-8f71-4213-a0c1-e826f3b84479 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.848686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30b94e-0bb4-45b4-a16d-3e964b3145b8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.905294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2e80c-f9d1-4faf-8d10-7f7c1c2262ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.906767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2e80c-f9d1-4faf-8d10-7f7c1c2262ba None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.908263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2e80c-f9d1-4faf-8d10-7f7c1c2262ba None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.908817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2e80c-f9d1-4faf-8d10-7f7c1c2262ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.910373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2e80c-f9d1-4faf-8d10-7f7c1c2262ba None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=e38a311f2b614a3598f6b51b3a235178) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:22.911427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2e80c-f9d1-4faf-8d10-7f7c1c2262ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.913165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2e80c-f9d1-4faf-8d10-7f7c1c2262ba None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:22.920690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 349/687] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:22 2022] GET /identity/v3/role_assignments?user.id=e38a311f2b614a3598f6b51b3a235178&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:22.921748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2e80c-f9d1-4faf-8d10-7f7c1c2262ba None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:22.924351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96b16b-8f71-4213-a0c1-e826f3b84479 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:22.957053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:22.980928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dd369-76ba-4f93-91c8-29de1216221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:22.981979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dd369-76ba-4f93-91c8-29de1216221b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:22.982419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dd369-76ba-4f93-91c8-29de1216221b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:22.982853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dd369-76ba-4f93-91c8-29de1216221b None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:22.983989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dd369-76ba-4f93-91c8-29de1216221b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:22.984699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dd369-76ba-4f93-91c8-29de1216221b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:22.986344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dd369-76ba-4f93-91c8-29de1216221b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:23.010497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 339/688] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:23.016346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dd369-76ba-4f93-91c8-29de1216221b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:23.072414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:23.076055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2e80c-f9d1-4faf-8d10-7f7c1c2262ba None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:23.124949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18299dcd-e203-483b-b590-87ade0322ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:23.124949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18299dcd-e203-483b-b590-87ade0322ea5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:23.124949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18299dcd-e203-483b-b590-87ade0322ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:23.124949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18299dcd-e203-483b-b590-87ade0322ea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:23.158728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dd369-76ba-4f93-91c8-29de1216221b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:23.200067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a3733-081e-446e-bcba-656a578518cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:23.201051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a3733-081e-446e-bcba-656a578518cc None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:23.201501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a3733-081e-446e-bcba-656a578518cc None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:23.201924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a3733-081e-446e-bcba-656a578518cc None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:23.203496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a3733-081e-446e-bcba-656a578518cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:23.205247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a3733-081e-446e-bcba-656a578518cc None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:23.214036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18299dcd-e203-483b-b590-87ade0322ea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:23.214036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18299dcd-e203-483b-b590-87ade0322ea5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:23.232768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 340/689] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:36:23 2022] POST /identity/v3/users => generated 254 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:23.237632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:23.330000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 350/690] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:23.335647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a3733-081e-446e-bcba-656a578518cc None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:23.469363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:23.473419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a3733-081e-446e-bcba-656a578518cc None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:23.513240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332de1f-24cf-4120-9e3b-574177812e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:23.514298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332de1f-24cf-4120-9e3b-574177812e1c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:23.514776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332de1f-24cf-4120-9e3b-574177812e1c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:23.515268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332de1f-24cf-4120-9e3b-574177812e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:23.527065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332de1f-24cf-4120-9e3b-574177812e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:23.528781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332de1f-24cf-4120-9e3b-574177812e1c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-a332de1f-24cf-4120-9e3b-574177812e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:23.547210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:23.548304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:23.549397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:23.549397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:23.549397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:23.549397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:23.551261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 341/691] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:23.556897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18299dcd-e203-483b-b590-87ade0322ea5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:23.682487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18299dcd-e203-483b-b590-87ade0322ea5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:23.703351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:23.733530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c591e50-57c9-4b35-9ba3-86f4bf346683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:23.734475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c591e50-57c9-4b35-9ba3-86f4bf346683 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:23.734475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c591e50-57c9-4b35-9ba3-86f4bf346683 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:23.734834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c591e50-57c9-4b35-9ba3-86f4bf346683 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:23.735586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c591e50-57c9-4b35-9ba3-86f4bf346683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:23.735897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c591e50-57c9-4b35-9ba3-86f4bf346683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:23.737326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c591e50-57c9-4b35-9ba3-86f4bf346683 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:23.759068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 351/692] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:23.764674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332de1f-24cf-4120-9e3b-574177812e1c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:23.899686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332de1f-24cf-4120-9e3b-574177812e1c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:23.953842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c31b78-d582-4b2f-bbc1-32e2bf08ec3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:23.954993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c31b78-d582-4b2f-bbc1-32e2bf08ec3d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:23.955478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c31b78-d582-4b2f-bbc1-32e2bf08ec3d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:23.955953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c31b78-d582-4b2f-bbc1-32e2bf08ec3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:23.964356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c31b78-d582-4b2f-bbc1-32e2bf08ec3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:23.966102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c31b78-d582-4b2f-bbc1-32e2bf08ec3d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-f2c31b78-d582-4b2f-bbc1-32e2bf08ec3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:23.981518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:23.982382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:23.983328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:23.983328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:23.983328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:23.983328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:23.983328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:23.983328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:23.983328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:23.983328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:23.986754 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 342/693] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:23.993652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c591e50-57c9-4b35-9ba3-86f4bf346683 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:24.005588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc428ae-1b55-4fe0-8ad5-e3bb1aab13bf None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:24.006183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc428ae-1b55-4fe0-8ad5-e3bb1aab13bf None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:24.006658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc428ae-1b55-4fe0-8ad5-e3bb1aab13bf None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:24.008196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 343/694] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:24.016696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880ff03-5a03-46f5-87a2-6997b327a752 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:24.016981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880ff03-5a03-46f5-87a2-6997b327a752 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:24.017256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880ff03-5a03-46f5-87a2-6997b327a752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:24.047420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3880ff03-5a03-46f5-87a2-6997b327a752 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:24.048634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3880ff03-5a03-46f5-87a2-6997b327a752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:24.063546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:24.151585 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 344/695] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:36:24.175906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880ff03-5a03-46f5-87a2-6997b327a752 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:24.186818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c591e50-57c9-4b35-9ba3-86f4bf346683 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:24.258707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ca780-8223-4aa3-8e58-64e831f7a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:24.259301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ca780-8223-4aa3-8e58-64e831f7a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:24.259585 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ca780-8223-4aa3-8e58-64e831f7a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:24.259864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ca780-8223-4aa3-8e58-64e831f7a4f3 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:24.260684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ca780-8223-4aa3-8e58-64e831f7a4f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:24.260995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ca780-8223-4aa3-8e58-64e831f7a4f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:24.262413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ca780-8223-4aa3-8e58-64e831f7a4f3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:24.291673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 352/696] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:24.293140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ca780-8223-4aa3-8e58-64e831f7a4f3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:24.346574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880ff03-5a03-46f5-87a2-6997b327a752 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:24.374544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:24.387546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788adbbe-11fd-4088-9a02-ca683ce392bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:24.388541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788adbbe-11fd-4088-9a02-ca683ce392bf None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:24.389205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788adbbe-11fd-4088-9a02-ca683ce392bf None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:24.389787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788adbbe-11fd-4088-9a02-ca683ce392bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:24.402711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788adbbe-11fd-4088-9a02-ca683ce392bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-788adbbe-11fd-4088-9a02-ca683ce392bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:24.412118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:24.413285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:24.414324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:24.420789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 345/697] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:24.423016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788adbbe-11fd-4088-9a02-ca683ce392bf None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:24.426141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:24.538191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788adbbe-11fd-4088-9a02-ca683ce392bf None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:24.563977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ca780-8223-4aa3-8e58-64e831f7a4f3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:24.575415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4561b8-4b1e-4f3d-9d51-6ae1eacc58da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:24.576272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4561b8-4b1e-4f3d-9d51-6ae1eacc58da None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:24.576751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4561b8-4b1e-4f3d-9d51-6ae1eacc58da None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:24.577409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4561b8-4b1e-4f3d-9d51-6ae1eacc58da None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:24.578566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4561b8-4b1e-4f3d-9d51-6ae1eacc58da None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:24.579023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4561b8-4b1e-4f3d-9d51-6ae1eacc58da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:24.580665 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4561b8-4b1e-4f3d-9d51-6ae1eacc58da None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:24.589775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 346/698] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:24.595938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4561b8-4b1e-4f3d-9d51-6ae1eacc58da None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:24.651505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f28c7-9f95-4ce7-b348-cf0688a68fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:24.652607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f28c7-9f95-4ce7-b348-cf0688a68fa0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:24.652726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f28c7-9f95-4ce7-b348-cf0688a68fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:24.653131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f28c7-9f95-4ce7-b348-cf0688a68fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:24.654135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f28c7-9f95-4ce7-b348-cf0688a68fa0 None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=3dd1e02af2374224b968c2644dd0c813) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:24.654553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f28c7-9f95-4ce7-b348-cf0688a68fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:24.656856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f28c7-9f95-4ce7-b348-cf0688a68fa0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:24.675406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 353/699] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:24 2022] GET /identity/v3/role_assignments?user.id=3dd1e02af2374224b968c2644dd0c813&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 608 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:24.680026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:24.724534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4561b8-4b1e-4f3d-9d51-6ae1eacc58da None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:24.762288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f856ec8-e9aa-4656-974e-c9333c7dff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:24.763410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f856ec8-e9aa-4656-974e-c9333c7dff8e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:24.763843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f856ec8-e9aa-4656-974e-c9333c7dff8e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:24.764284 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f856ec8-e9aa-4656-974e-c9333c7dff8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:24.778401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f856ec8-e9aa-4656-974e-c9333c7dff8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:24.781160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f856ec8-e9aa-4656-974e-c9333c7dff8e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-4f856ec8-e9aa-4656-974e-c9333c7dff8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:24.802008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:24.803076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:24.804151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:24.804151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:24.804151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:24.804151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:24.806356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 347/700] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:24.813761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f28c7-9f95-4ce7-b348-cf0688a68fa0 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:24.942287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f28c7-9f95-4ce7-b348-cf0688a68fa0 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:24.944023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:24.981073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5ba9d-e21a-48d3-b7a6-226eafe52bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:24.981789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5ba9d-e21a-48d3-b7a6-226eafe52bf8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:24.982078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5ba9d-e21a-48d3-b7a6-226eafe52bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:24.982360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5ba9d-e21a-48d3-b7a6-226eafe52bf8 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:24.983090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5ba9d-e21a-48d3-b7a6-226eafe52bf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:24.983900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5ba9d-e21a-48d3-b7a6-226eafe52bf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:24.985382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5ba9d-e21a-48d3-b7a6-226eafe52bf8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:24.997362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 354/701] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:25.003475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f856ec8-e9aa-4656-974e-c9333c7dff8e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:25.114747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f856ec8-e9aa-4656-974e-c9333c7dff8e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:25.132369 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:25.151452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d809aa-e257-4caa-ad95-5b591d2da2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:25.152264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d809aa-e257-4caa-ad95-5b591d2da2a5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.152551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d809aa-e257-4caa-ad95-5b591d2da2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.152861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d809aa-e257-4caa-ad95-5b591d2da2a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.161237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d809aa-e257-4caa-ad95-5b591d2da2a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:25.162694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d809aa-e257-4caa-ad95-5b591d2da2a5 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-a0d809aa-e257-4caa-ad95-5b591d2da2a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:25.177353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:25.178277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:25.179200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 348/702] 10.222.0.47 () {56 vars in 1093 bytes} [Fri Jun 24 23:36:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:25.184366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5ba9d-e21a-48d3-b7a6-226eafe52bf8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:25.297661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5ba9d-e21a-48d3-b7a6-226eafe52bf8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:25.336217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d228ade-d0f8-4c7b-8030-3049c43aca3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:25.336931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d228ade-d0f8-4c7b-8030-3049c43aca3e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.337266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d228ade-d0f8-4c7b-8030-3049c43aca3e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.337556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d228ade-d0f8-4c7b-8030-3049c43aca3e None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.338416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d228ade-d0f8-4c7b-8030-3049c43aca3e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:25.338731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d228ade-d0f8-4c7b-8030-3049c43aca3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:25.340182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d228ade-d0f8-4c7b-8030-3049c43aca3e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:25.361897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 355/703] 10.222.0.47 () {56 vars in 1133 bytes} [Fri Jun 24 23:36:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:25.367522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d809aa-e257-4caa-ad95-5b591d2da2a5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:25.385609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6ee27-e871-4aeb-b2ac-9b73a9ebdf07 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.385609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6ee27-e871-4aeb-b2ac-9b73a9ebdf07 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.385609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6ee27-e871-4aeb-b2ac-9b73a9ebdf07 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.387696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 356/704] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:25.393311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5304-1b5b-468c-8615-c967b6181df8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.393785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5304-1b5b-468c-8615-c967b6181df8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.394398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a5304-1b5b-468c-8615-c967b6181df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.422731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-521a5304-1b5b-468c-8615-c967b6181df8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:25.424868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-521a5304-1b5b-468c-8615-c967b6181df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:25.460395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:25.540664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d809aa-e257-4caa-ad95-5b591d2da2a5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:25.557774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 357/705] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 23:36:25.595897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a5304-1b5b-468c-8615-c967b6181df8 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:25.625470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469ec0f-ba09-4d7d-a6c7-d4f72fbf3eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:25.626584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469ec0f-ba09-4d7d-a6c7-d4f72fbf3eec None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.627031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469ec0f-ba09-4d7d-a6c7-d4f72fbf3eec None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.627535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469ec0f-ba09-4d7d-a6c7-d4f72fbf3eec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.718793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:25.721244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a5304-1b5b-468c-8615-c967b6181df8 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:25.756985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01be47-0969-44a2-b513-815fd9ff2c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:25.758077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01be47-0969-44a2-b513-815fd9ff2c07 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.758411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469ec0f-ba09-4d7d-a6c7-d4f72fbf3eec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:25.758657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01be47-0969-44a2-b513-815fd9ff2c07 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.759067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01be47-0969-44a2-b513-815fd9ff2c07 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.759860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01be47-0969-44a2-b513-815fd9ff2c07 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:25.759983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469ec0f-ba09-4d7d-a6c7-d4f72fbf3eec None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:25.761490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01be47-0969-44a2-b513-815fd9ff2c07 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-2a01be47-0969-44a2-b513-815fd9ff2c07 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:25.776932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:25.777931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 23:36:25.778756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:25.781783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 358/706] 10.222.0.47 () {56 vars in 1129 bytes} [Fri Jun 24 23:36:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:25.783264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4c49-d2fe-4390-8818-738238919fd5 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.783726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4c49-d2fe-4390-8818-738238919fd5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.784143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a4c49-d2fe-4390-8818-738238919fd5 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.792631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 359/707] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:25.792631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4c49-d2fe-4390-8818-738238919fd5 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:25.809871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:25.885155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 349/708] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:25.886451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df86ed1-b93b-4565-92dd-632b01ae4548 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.886741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df86ed1-b93b-4565-92dd-632b01ae4548 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.887023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df86ed1-b93b-4565-92dd-632b01ae4548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.905103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7df86ed1-b93b-4565-92dd-632b01ae4548 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:25.906865 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7df86ed1-b93b-4565-92dd-632b01ae4548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:25.929882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a4c49-d2fe-4390-8818-738238919fd5 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:25.968643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d220f75-9430-4a15-a0f4-e00f0e20872e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:25.969422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d220f75-9430-4a15-a0f4-e00f0e20872e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.969719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d220f75-9430-4a15-a0f4-e00f0e20872e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.970004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d220f75-9430-4a15-a0f4-e00f0e20872e None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.970721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d220f75-9430-4a15-a0f4-e00f0e20872e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:25.971036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d220f75-9430-4a15-a0f4-e00f0e20872e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:25.972425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d220f75-9430-4a15-a0f4-e00f0e20872e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:25.977267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 360/709] 10.222.0.47 () {56 vars in 1124 bytes} [Fri Jun 24 23:36:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:25.978370 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146d60-349d-4226-8bd8-b6c50022d224 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:25.978660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146d60-349d-4226-8bd8-b6c50022d224 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:25.978939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146d60-349d-4226-8bd8-b6c50022d224 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:25.980298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 361/710] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:25.985830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146d60-349d-4226-8bd8-b6c50022d224 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:25.989663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 350/711] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:25.990684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df86ed1-b93b-4565-92dd-632b01ae4548 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:26.006331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:26.081609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:26.105488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146d60-349d-4226-8bd8-b6c50022d224 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.105910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df86ed1-b93b-4565-92dd-632b01ae4548 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.140661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0844c4-09a1-48de-877b-2b00a98cbdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:26.141265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0844c4-09a1-48de-877b-2b00a98cbdab None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.141553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0844c4-09a1-48de-877b-2b00a98cbdab None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.141832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0844c4-09a1-48de-877b-2b00a98cbdab None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.142549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0844c4-09a1-48de-877b-2b00a98cbdab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:26.142890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0844c4-09a1-48de-877b-2b00a98cbdab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:26.143490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86046423-a170-486c-942a-dfa2a1541836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:26.144240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0844c4-09a1-48de-877b-2b00a98cbdab None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:26.144337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86046423-a170-486c-942a-dfa2a1541836 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.144554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86046423-a170-486c-942a-dfa2a1541836 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.144953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86046423-a170-486c-942a-dfa2a1541836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.150523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 351/712] 10.222.0.47 () {56 vars in 1124 bytes} [Fri Jun 24 23:36:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:26.151551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766b374-73c6-4445-9f42-5fc3d29c9e1f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.151841 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766b374-73c6-4445-9f42-5fc3d29c9e1f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.152123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766b374-73c6-4445-9f42-5fc3d29c9e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.166017 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86046423-a170-486c-942a-dfa2a1541836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:26.168216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86046423-a170-486c-942a-dfa2a1541836 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:26.171928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-e766b374-73c6-4445-9f42-5fc3d29c9e1f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:26.173203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e766b374-73c6-4445-9f42-5fc3d29c9e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-86046423-a170-486c-942a-dfa2a1541836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:26.197408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:26.199182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:26.208000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:26.208000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:26.208000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:26.208000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:26.208000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 362/713] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:26.208000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86046423-a170-486c-942a-dfa2a1541836 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:26.278695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:26.287390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 352/714] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:26.288718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766b374-73c6-4445-9f42-5fc3d29c9e1f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:26.351052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86046423-a170-486c-942a-dfa2a1541836 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.384455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:26.392008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c71c4-1b9a-4738-8e48-e4293509e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:26.392863 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c71c4-1b9a-4738-8e48-e4293509e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.393213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c71c4-1b9a-4738-8e48-e4293509e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.393557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c71c4-1b9a-4738-8e48-e4293509e7e0 None admin] PATH_INFO: `/v3/services/network` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.394294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c71c4-1b9a-4738-8e48-e4293509e7e0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:26.395738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c71c4-1b9a-4738-8e48-e4293509e7e0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-7c7c71c4-1b9a-4738-8e48-e4293509e7e0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:26.409042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:26.410242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 23:36:26.411328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:26.416718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 363/715] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:26 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:26.417614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c71c4-1b9a-4738-8e48-e4293509e7e0 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:26.422841 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766b374-73c6-4445-9f42-5fc3d29c9e1f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.472667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07bc97-91d4-4975-adcf-c3557f1fc20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:26.473348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07bc97-91d4-4975-adcf-c3557f1fc20d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.473629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07bc97-91d4-4975-adcf-c3557f1fc20d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.473901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07bc97-91d4-4975-adcf-c3557f1fc20d None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.474696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07bc97-91d4-4975-adcf-c3557f1fc20d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:26.475004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07bc97-91d4-4975-adcf-c3557f1fc20d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:26.476430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07bc97-91d4-4975-adcf-c3557f1fc20d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:26.489711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 353/716] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:26.490606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07bc97-91d4-4975-adcf-c3557f1fc20d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:26.540913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c71c4-1b9a-4738-8e48-e4293509e7e0 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.556377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:26.579511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace11549-167d-41b7-be56-f2f6fc199329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:26.580318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace11549-167d-41b7-be56-f2f6fc199329 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.580617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace11549-167d-41b7-be56-f2f6fc199329 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.581006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace11549-167d-41b7-be56-f2f6fc199329 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.592323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace11549-167d-41b7-be56-f2f6fc199329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:26.600935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07bc97-91d4-4975-adcf-c3557f1fc20d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ace11549-167d-41b7-be56-f2f6fc199329 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:26.601150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:26.603226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 364/717] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:26.605283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace11549-167d-41b7-be56-f2f6fc199329 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:26.642290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c14002-d3bf-43e6-8cf4-f98b00d083d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:26.642941 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c14002-d3bf-43e6-8cf4-f98b00d083d1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.643229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c14002-d3bf-43e6-8cf4-f98b00d083d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.643560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c14002-d3bf-43e6-8cf4-f98b00d083d1 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.644341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c14002-d3bf-43e6-8cf4-f98b00d083d1 None admin] RBAC: Adding query filter params (name=network) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:26.644684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c14002-d3bf-43e6-8cf4-f98b00d083d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:26.646122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c14002-d3bf-43e6-8cf4-f98b00d083d1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:26.656194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 354/718] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:26.657148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c14002-d3bf-43e6-8cf4-f98b00d083d1 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:26.665023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:26.715913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace11549-167d-41b7-be56-f2f6fc199329 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.753477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe0f35-9d65-4a09-be55-5082292845a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:26.754377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe0f35-9d65-4a09-be55-5082292845a2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.754708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe0f35-9d65-4a09-be55-5082292845a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.754937 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe0f35-9d65-4a09-be55-5082292845a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.762334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe0f35-9d65-4a09-be55-5082292845a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:26.762654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c14002-d3bf-43e6-8cf4-f98b00d083d1 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.763803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe0f35-9d65-4a09-be55-5082292845a2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-67fe0f35-9d65-4a09-be55-5082292845a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:26.779707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:26.782853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 365/719] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:26.794916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe0f35-9d65-4a09-be55-5082292845a2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:26.805334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:26.805334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d353a9-ce4b-4d44-81f3-036b6b9b5d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:26.805334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d353a9-ce4b-4d44-81f3-036b6b9b5d31 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.805334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d353a9-ce4b-4d44-81f3-036b6b9b5d31 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.805917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d353a9-ce4b-4d44-81f3-036b6b9b5d31 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.805917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d353a9-ce4b-4d44-81f3-036b6b9b5d31 None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:26.805917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d353a9-ce4b-4d44-81f3-036b6b9b5d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:26.805917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d353a9-ce4b-4d44-81f3-036b6b9b5d31 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:26.818891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 355/720] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:26.819713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d353a9-ce4b-4d44-81f3-036b6b9b5d31 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:26.905545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe0f35-9d65-4a09-be55-5082292845a2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.935765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:26.953338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d353a9-ce4b-4d44-81f3-036b6b9b5d31 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:26.964839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acfbbb-b5c7-401a-967a-4728ee4c222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:26.965517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acfbbb-b5c7-401a-967a-4728ee4c222c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:26.965804 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acfbbb-b5c7-401a-967a-4728ee4c222c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:26.966086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acfbbb-b5c7-401a-967a-4728ee4c222c None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:26.966812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acfbbb-b5c7-401a-967a-4728ee4c222c None admin] RBAC: Adding query filter params (type=network) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:26.967125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acfbbb-b5c7-401a-967a-4728ee4c222c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:26.969368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acfbbb-b5c7-401a-967a-4728ee4c222c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:26.980670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 366/721] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:26.980670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acfbbb-b5c7-401a-967a-4728ee4c222c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:27.025175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155cffd-8b8e-4379-85d2-e4a44083e347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:27.027097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155cffd-8b8e-4379-85d2-e4a44083e347 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:27.027533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155cffd-8b8e-4379-85d2-e4a44083e347 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:27.027958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155cffd-8b8e-4379-85d2-e4a44083e347 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:27.029640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155cffd-8b8e-4379-85d2-e4a44083e347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:27.030338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155cffd-8b8e-4379-85d2-e4a44083e347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:27.033387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155cffd-8b8e-4379-85d2-e4a44083e347 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:27.062884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 356/722] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:27.067355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155cffd-8b8e-4379-85d2-e4a44083e347 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:27.143523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acfbbb-b5c7-401a-967a-4728ee4c222c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:27.172705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:27.181013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc662d-0d6c-4d94-bd1f-d2906622c815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:27.182012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc662d-0d6c-4d94-bd1f-d2906622c815 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:27.182467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc662d-0d6c-4d94-bd1f-d2906622c815 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:27.182911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc662d-0d6c-4d94-bd1f-d2906622c815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:27.194751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc662d-0d6c-4d94-bd1f-d2906622c815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:27.196493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc662d-0d6c-4d94-bd1f-d2906622c815 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-3ecc662d-0d6c-4d94-bd1f-d2906622c815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:27.214539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:27.215590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:27.216674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:27.216674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:27.216674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:27.216674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:27.218644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 367/723] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:27.224000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc662d-0d6c-4d94-bd1f-d2906622c815 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:27.242769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155cffd-8b8e-4379-85d2-e4a44083e347 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:27.275062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:27.292322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a44937-34aa-44d2-9f89-293c8974973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:27.293381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a44937-34aa-44d2-9f89-293c8974973e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:27.293903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a44937-34aa-44d2-9f89-293c8974973e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:27.294415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a44937-34aa-44d2-9f89-293c8974973e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:27.295512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a44937-34aa-44d2-9f89-293c8974973e None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=feb9bec9f1894e179e24a14b81b3f23d) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:27.296143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a44937-34aa-44d2-9f89-293c8974973e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:27.299356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a44937-34aa-44d2-9f89-293c8974973e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:27.305701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 357/724] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:27 2022] GET /identity/v3/role_assignments?user.id=feb9bec9f1894e179e24a14b81b3f23d&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:27.366319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc662d-0d6c-4d94-bd1f-d2906622c815 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:27.380231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa763a4e-ad1d-4c53-8663-3b595639bb5d None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:27.380555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa763a4e-ad1d-4c53-8663-3b595639bb5d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:27.380832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa763a4e-ad1d-4c53-8663-3b595639bb5d None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:27.382342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 358/725] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:27.388301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42b9ef-aa2a-4345-90dd-ee31d16870b6 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:27.388630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42b9ef-aa2a-4345-90dd-ee31d16870b6 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:27.388904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42b9ef-aa2a-4345-90dd-ee31d16870b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:27.424247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-8e42b9ef-aa2a-4345-90dd-ee31d16870b6 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:27.426231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8e42b9ef-aa2a-4345-90dd-ee31d16870b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:27.432540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c84c0-b76f-48b8-a5e1-2849a4d846a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:27.433629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c84c0-b76f-48b8-a5e1-2849a4d846a7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:27.434119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c84c0-b76f-48b8-a5e1-2849a4d846a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:27.434618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c84c0-b76f-48b8-a5e1-2849a4d846a7 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:27.435611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c84c0-b76f-48b8-a5e1-2849a4d846a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:27.445242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c84c0-b76f-48b8-a5e1-2849a4d846a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:27.446592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c84c0-b76f-48b8-a5e1-2849a4d846a7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:27.472516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 368/726] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:27.478809 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c84c0-b76f-48b8-a5e1-2849a4d846a7 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:27.521145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 359/727] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:27.546437 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42b9ef-aa2a-4345-90dd-ee31d16870b6 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:27.633512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:27.664981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:27.679463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c84c0-b76f-48b8-a5e1-2849a4d846a7 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:27.734249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c663f-a482-4551-8e84-dd1804fdb439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:27.736162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c663f-a482-4551-8e84-dd1804fdb439 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:27.736446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c663f-a482-4551-8e84-dd1804fdb439 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:27.736760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c663f-a482-4551-8e84-dd1804fdb439 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:27.748524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42b9ef-aa2a-4345-90dd-ee31d16870b6 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:27.752246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c663f-a482-4551-8e84-dd1804fdb439 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:27.753749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c663f-a482-4551-8e84-dd1804fdb439 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-7a0c663f-a482-4551-8e84-dd1804fdb439 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:27.780492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:27.781524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:27.782442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 369/728] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:36:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:27.790928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c663f-a482-4551-8e84-dd1804fdb439 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:27.816250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f1169-1d7e-4a84-812b-e0b491e2d3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:27.817224 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f1169-1d7e-4a84-812b-e0b491e2d3cf None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:27.817511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f1169-1d7e-4a84-812b-e0b491e2d3cf None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:27.817790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f1169-1d7e-4a84-812b-e0b491e2d3cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:27.830798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f1169-1d7e-4a84-812b-e0b491e2d3cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-012f1169-1d7e-4a84-812b-e0b491e2d3cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:27.839288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:27.840179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:27.841029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:27.841679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 360/729] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:27.848257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f1169-1d7e-4a84-812b-e0b491e2d3cf None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:27.939650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c663f-a482-4551-8e84-dd1804fdb439 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:27.989321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ee92c-32c5-4447-a776-4186a5910c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:27.990216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ee92c-32c5-4447-a776-4186a5910c7f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:27.990507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ee92c-32c5-4447-a776-4186a5910c7f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:27.990790 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ee92c-32c5-4447-a776-4186a5910c7f None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:27.991649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ee92c-32c5-4447-a776-4186a5910c7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:27.991961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ee92c-32c5-4447-a776-4186a5910c7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:27.993415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ee92c-32c5-4447-a776-4186a5910c7f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:28.025821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 370/730] 10.222.0.47 () {56 vars in 1149 bytes} [Fri Jun 24 23:36:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:28.027041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ee92c-32c5-4447-a776-4186a5910c7f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:28.035395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:28.038641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f1169-1d7e-4a84-812b-e0b491e2d3cf None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:28.043096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:28.076307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67573810-4a03-4a88-80d1-edf7fac39806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:28.077503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67573810-4a03-4a88-80d1-edf7fac39806 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:28.077859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67573810-4a03-4a88-80d1-edf7fac39806 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:28.078146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67573810-4a03-4a88-80d1-edf7fac39806 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:28.078918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67573810-4a03-4a88-80d1-edf7fac39806 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:28.079231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67573810-4a03-4a88-80d1-edf7fac39806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:28.080632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67573810-4a03-4a88-80d1-edf7fac39806 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:28.091336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 361/731] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:28.097553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67573810-4a03-4a88-80d1-edf7fac39806 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:28.205647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ee92c-32c5-4447-a776-4186a5910c7f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:28.269978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed832dda-bf93-43c8-8bc4-168f91b39275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:28.270642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed832dda-bf93-43c8-8bc4-168f91b39275 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:28.270924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed832dda-bf93-43c8-8bc4-168f91b39275 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:28.271433 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed832dda-bf93-43c8-8bc4-168f91b39275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:28.294971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67573810-4a03-4a88-80d1-edf7fac39806 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:28.336424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab37c2d-3dac-4172-a01a-1c46ddb59757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:28.337422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab37c2d-3dac-4172-a01a-1c46ddb59757 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:28.337706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab37c2d-3dac-4172-a01a-1c46ddb59757 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:28.337988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab37c2d-3dac-4172-a01a-1c46ddb59757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:28.360706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab37c2d-3dac-4172-a01a-1c46ddb59757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:28.362375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab37c2d-3dac-4172-a01a-1c46ddb59757 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-3ab37c2d-3dac-4172-a01a-1c46ddb59757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:28.393773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:28.395921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:28.398051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:28.398051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:28.398051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:28.398051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:28.398051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 362/732] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:28.405494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab37c2d-3dac-4172-a01a-1c46ddb59757 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:28.411095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed832dda-bf93-43c8-8bc4-168f91b39275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:28.412559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed832dda-bf93-43c8-8bc4-168f91b39275 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:28.433323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:28.439121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:28.549619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 371/733] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 23:36:28.551413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39fefd-7821-4bcc-aca2-ec7d73010821 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:28.551928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab37c2d-3dac-4172-a01a-1c46ddb59757 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:28.552039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39fefd-7821-4bcc-aca2-ec7d73010821 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:28.552549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39fefd-7821-4bcc-aca2-ec7d73010821 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:28.558510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 372/734] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:28.559627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39fefd-7821-4bcc-aca2-ec7d73010821 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:28.598482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6c61c-ac93-4740-9354-658a8c88cc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:28.599152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6c61c-ac93-4740-9354-658a8c88cc63 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:28.599432 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6c61c-ac93-4740-9354-658a8c88cc63 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:28.599705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6c61c-ac93-4740-9354-658a8c88cc63 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:28.600416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6c61c-ac93-4740-9354-658a8c88cc63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:28.600774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6c61c-ac93-4740-9354-658a8c88cc63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:28.602203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6c61c-ac93-4740-9354-658a8c88cc63 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:28.619933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 363/735] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:28.621658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151a0d8-7ee9-492a-b64c-c3a5243956ae None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:28.621888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151a0d8-7ee9-492a-b64c-c3a5243956ae None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:28.622132 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151a0d8-7ee9-492a-b64c-c3a5243956ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:28.639294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1151a0d8-7ee9-492a-b64c-c3a5243956ae None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:28.641151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1151a0d8-7ee9-492a-b64c-c3a5243956ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:28.689407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39fefd-7821-4bcc-aca2-ec7d73010821 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:28.699341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:28.727725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bf324-77c8-46cf-8115-f4a83f828e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:28.728557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 364/736] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:28.729645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151a0d8-7ee9-492a-b64c-c3a5243956ae None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:28.732887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bf324-77c8-46cf-8115-f4a83f828e25 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:28.734678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bf324-77c8-46cf-8115-f4a83f828e25 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:28.735325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bf324-77c8-46cf-8115-f4a83f828e25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:28.740913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:28.755303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bf324-77c8-46cf-8115-f4a83f828e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:28.756799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bf324-77c8-46cf-8115-f4a83f828e25 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-367bf324-77c8-46cf-8115-f4a83f828e25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:28.775421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:28.776286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:28.777195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:28.777195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:28.777195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:28.777195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:28.777195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 373/737] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:28.778590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bf324-77c8-46cf-8115-f4a83f828e25 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:28.873795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151a0d8-7ee9-492a-b64c-c3a5243956ae None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:28.891435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bf324-77c8-46cf-8115-f4a83f828e25 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:28.910208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b7e93-54a9-4747-9e90-c16fc303ea21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:28.911206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b7e93-54a9-4747-9e90-c16fc303ea21 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:28.911664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b7e93-54a9-4747-9e90-c16fc303ea21 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:28.912079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b7e93-54a9-4747-9e90-c16fc303ea21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:28.920008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b7e93-54a9-4747-9e90-c16fc303ea21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:28.922413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b7e93-54a9-4747-9e90-c16fc303ea21 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:28.929777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbfd0c-42f9-48a8-aab5-8cba7a531e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:28.935883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbfd0c-42f9-48a8-aab5-8cba7a531e32 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:28.936260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbfd0c-42f9-48a8-aab5-8cba7a531e32 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:28.936735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbfd0c-42f9-48a8-aab5-8cba7a531e32 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-3e9b7e93-54a9-4747-9e90-c16fc303ea21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:28.937711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:28.938581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:28.939475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:28.939982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbfd0c-42f9-48a8-aab5-8cba7a531e32 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:28.939982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbfd0c-42f9-48a8-aab5-8cba7a531e32 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:28.945975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 365/738] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:28.946261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b7e93-54a9-4747-9e90-c16fc303ea21 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:28.966783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 374/739] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:36:28 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:28.966783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbfd0c-42f9-48a8-aab5-8cba7a531e32 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:29.021770 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:29.078305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:29.151159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b7e93-54a9-4747-9e90-c16fc303ea21 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:29.190117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbfd0c-42f9-48a8-aab5-8cba7a531e32 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:29.197348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1066e-3edc-4546-b09b-95fa95b0c15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:29.200963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1066e-3edc-4546-b09b-95fa95b0c15e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.201480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1066e-3edc-4546-b09b-95fa95b0c15e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.201955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1066e-3edc-4546-b09b-95fa95b0c15e None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.203356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1066e-3edc-4546-b09b-95fa95b0c15e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:29.204211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1066e-3edc-4546-b09b-95fa95b0c15e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:29.206296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1066e-3edc-4546-b09b-95fa95b0c15e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:29.221909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 366/740] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:29.227202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1066e-3edc-4546-b09b-95fa95b0c15e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:29.235643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0a632-d66e-4adf-851a-5ade397bff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:29.236710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0a632-d66e-4adf-851a-5ade397bff16 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.237177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0a632-d66e-4adf-851a-5ade397bff16 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.237635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0a632-d66e-4adf-851a-5ade397bff16 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.238691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0a632-d66e-4adf-851a-5ade397bff16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:29.239640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0a632-d66e-4adf-851a-5ade397bff16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:29.241823 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0a632-d66e-4adf-851a-5ade397bff16 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:29.266067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 375/741] 10.222.0.47 () {56 vars in 1135 bytes} [Fri Jun 24 23:36:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:29.271810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0a632-d66e-4adf-851a-5ade397bff16 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:29.347301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1066e-3edc-4546-b09b-95fa95b0c15e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:29.356468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:29.372775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:29.385245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4894e-c270-434b-a354-243fc2068b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:29.386061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4894e-c270-434b-a354-243fc2068b9f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.386347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4894e-c270-434b-a354-243fc2068b9f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.386629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4894e-c270-434b-a354-243fc2068b9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.391818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0a632-d66e-4adf-851a-5ade397bff16 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:29.394338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4894e-c270-434b-a354-243fc2068b9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:29.395804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e4894e-c270-434b-a354-243fc2068b9f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-40e4894e-c270-434b-a354-243fc2068b9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:29.411512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:29.412389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:29.413321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:29.413321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:29.413321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:29.413321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:29.413321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:29.413321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:29.413321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:29.413321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:29.413321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 367/742] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:29.418378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4894e-c270-434b-a354-243fc2068b9f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:29.433275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3a99e-eb86-4613-b149-52bd31fc7744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:29.434243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3a99e-eb86-4613-b149-52bd31fc7744 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.434697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3a99e-eb86-4613-b149-52bd31fc7744 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.435150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3a99e-eb86-4613-b149-52bd31fc7744 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.510975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3a99e-eb86-4613-b149-52bd31fc7744 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:29.512772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3a99e-eb86-4613-b149-52bd31fc7744 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:29.531671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4894e-c270-434b-a354-243fc2068b9f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:29.570759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ae66e-1ede-4168-81bc-e8c88d9e1400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:29.571557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ae66e-1ede-4168-81bc-e8c88d9e1400 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.571956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ae66e-1ede-4168-81bc-e8c88d9e1400 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.572290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ae66e-1ede-4168-81bc-e8c88d9e1400 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.573203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ae66e-1ede-4168-81bc-e8c88d9e1400 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:29.573586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ae66e-1ede-4168-81bc-e8c88d9e1400 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:29.575103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ae66e-1ede-4168-81bc-e8c88d9e1400 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:29.589531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 368/743] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:29.590955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b6922-1063-4e10-b92b-727a8b906d5d None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.591320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b6922-1063-4e10-b92b-727a8b906d5d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.591675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b6922-1063-4e10-b92b-727a8b906d5d None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.597336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 369/744] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:29.597336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b6922-1063-4e10-b92b-727a8b906d5d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:29.622674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:29.649144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:29.660613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 376/745] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 23:36:29.660613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d3e6e-70c9-4a4a-b630-15e43f08cada None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.660613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d3e6e-70c9-4a4a-b630-15e43f08cada None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.660613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118d3e6e-70c9-4a4a-b630-15e43f08cada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.683031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-118d3e6e-70c9-4a4a-b630-15e43f08cada None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:29.684151 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-118d3e6e-70c9-4a4a-b630-15e43f08cada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:29.725776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b6922-1063-4e10-b92b-727a8b906d5d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:29.759208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 377/746] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 23:36:29.760097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d3e6e-70c9-4a4a-b630-15e43f08cada None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:29.779741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5136b-406e-4361-a6cd-30ba7f7222bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:29.780443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5136b-406e-4361-a6cd-30ba7f7222bb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.780765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5136b-406e-4361-a6cd-30ba7f7222bb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.781037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5136b-406e-4361-a6cd-30ba7f7222bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.781870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5136b-406e-4361-a6cd-30ba7f7222bb None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=063b3fbafe0949d1b09bd9b7b0d70131) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:29.782178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5136b-406e-4361-a6cd-30ba7f7222bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:29.783606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5136b-406e-4361-a6cd-30ba7f7222bb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:29.795544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 370/747] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:29 2022] GET /identity/v3/role_assignments?user.id=063b3fbafe0949d1b09bd9b7b0d70131&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:29.795544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249195b-7044-4c70-ab09-0f82b8d5c41e None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.795544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249195b-7044-4c70-ab09-0f82b8d5c41e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.795544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249195b-7044-4c70-ab09-0f82b8d5c41e None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.795544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 371/748] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:29.795544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249195b-7044-4c70-ab09-0f82b8d5c41e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:29.918388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118d3e6e-70c9-4a4a-b630-15e43f08cada None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:29.923250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:29.949641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249195b-7044-4c70-ab09-0f82b8d5c41e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:29.973411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:29.979185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030f142-3e5e-4e14-9756-63f65dbd10e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:29.981716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030f142-3e5e-4e14-9756-63f65dbd10e3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:29.982152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030f142-3e5e-4e14-9756-63f65dbd10e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:29.982579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030f142-3e5e-4e14-9756-63f65dbd10e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:29.998638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030f142-3e5e-4e14-9756-63f65dbd10e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:30.001492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030f142-3e5e-4e14-9756-63f65dbd10e3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:30.008287 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d11df-b546-4669-a56e-2d40cd85c195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:30.009301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d11df-b546-4669-a56e-2d40cd85c195 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:30.009680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d11df-b546-4669-a56e-2d40cd85c195 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:30.010066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d11df-b546-4669-a56e-2d40cd85c195 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:30.011403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d11df-b546-4669-a56e-2d40cd85c195 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:30.013359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d11df-b546-4669-a56e-2d40cd85c195 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-8030f142-3e5e-4e14-9756-63f65dbd10e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:30.025274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:30.026232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:30.027108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:30.027108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:30.027108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:30.027108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:30.033742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 378/749] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:30.035623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 372/750] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:36:29 2022] POST /identity/v3/services => generated 240 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:30.036730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b383c86-08a1-4fbd-ab49-4b3ff65d7c0e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:30.036730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b383c86-08a1-4fbd-ab49-4b3ff65d7c0e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:30.037450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d11df-b546-4669-a56e-2d40cd85c195 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:30.038013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b383c86-08a1-4fbd-ab49-4b3ff65d7c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:30.077178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-7b383c86-08a1-4fbd-ab49-4b3ff65d7c0e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:30.081180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7b383c86-08a1-4fbd-ab49-4b3ff65d7c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:30.227162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 379/751] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:30.238543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d11df-b546-4669-a56e-2d40cd85c195 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:30.250990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b383c86-08a1-4fbd-ab49-4b3ff65d7c0e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:30.279688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40035d-9c48-45f5-a9a8-69453f041a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:30.280872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40035d-9c48-45f5-a9a8-69453f041a6e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:30.281234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40035d-9c48-45f5-a9a8-69453f041a6e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:30.281515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40035d-9c48-45f5-a9a8-69453f041a6e None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:30.282567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40035d-9c48-45f5-a9a8-69453f041a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:30.283142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40035d-9c48-45f5-a9a8-69453f041a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:30.284715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40035d-9c48-45f5-a9a8-69453f041a6e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:30.299766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 373/752] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:30.305275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40035d-9c48-45f5-a9a8-69453f041a6e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:30.340440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:30.355046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:30.369967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b383c86-08a1-4fbd-ab49-4b3ff65d7c0e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:30.411521 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990c9aa-501c-423b-b7a6-4d4071d60000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:30.412318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990c9aa-501c-423b-b7a6-4d4071d60000 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:30.412680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990c9aa-501c-423b-b7a6-4d4071d60000 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:30.413057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990c9aa-501c-423b-b7a6-4d4071d60000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:30.423902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990c9aa-501c-423b-b7a6-4d4071d60000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:30.430020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40035d-9c48-45f5-a9a8-69453f041a6e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-9990c9aa-501c-423b-b7a6-4d4071d60000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:30.431850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:30.432773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:30.433617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 380/753] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:30.439956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990c9aa-501c-423b-b7a6-4d4071d60000 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:30.479987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5eca8d-ce92-4722-aee2-a56791eaed57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:30.480988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5eca8d-ce92-4722-aee2-a56791eaed57 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:30.481273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5eca8d-ce92-4722-aee2-a56791eaed57 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:30.481950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5eca8d-ce92-4722-aee2-a56791eaed57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:30.489866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5eca8d-ce92-4722-aee2-a56791eaed57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:30.491330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5eca8d-ce92-4722-aee2-a56791eaed57 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-eb5eca8d-ce92-4722-aee2-a56791eaed57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:30.508660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:30.510731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:30.512888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:30.512888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:30.512888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:30.512888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:30.512888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:30.512888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:30.512888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:30.512888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:30.512888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 374/754] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:30.521175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5eca8d-ce92-4722-aee2-a56791eaed57 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:30.623732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990c9aa-501c-423b-b7a6-4d4071d60000 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:30.664537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440eee3-3c8a-4c09-b071-77231cd8da98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:30.665236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440eee3-3c8a-4c09-b071-77231cd8da98 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:30.665515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440eee3-3c8a-4c09-b071-77231cd8da98 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:30.666025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440eee3-3c8a-4c09-b071-77231cd8da98 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:30.666792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440eee3-3c8a-4c09-b071-77231cd8da98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:30.670557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440eee3-3c8a-4c09-b071-77231cd8da98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:30.670557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440eee3-3c8a-4c09-b071-77231cd8da98 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:30.679334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 381/755] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:30.684293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440eee3-3c8a-4c09-b071-77231cd8da98 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:30.689315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:30.731663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:30.746993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5eca8d-ce92-4722-aee2-a56791eaed57 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:30.784000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde322-2b52-4108-8956-50915b636c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:30.784816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde322-2b52-4108-8956-50915b636c03 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:30.785186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde322-2b52-4108-8956-50915b636c03 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:30.785541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde322-2b52-4108-8956-50915b636c03 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:30.786397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cde322-2b52-4108-8956-50915b636c03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:30.786879 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cde322-2b52-4108-8956-50915b636c03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:30.788286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cde322-2b52-4108-8956-50915b636c03 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:30.796296 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440eee3-3c8a-4c09-b071-77231cd8da98 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:30.802912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 375/756] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:30.808738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde322-2b52-4108-8956-50915b636c03 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:30.831975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b8ef3-6258-40f9-8cb3-f4ce08f18f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:30.833121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b8ef3-6258-40f9-8cb3-f4ce08f18f91 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:30.833592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b8ef3-6258-40f9-8cb3-f4ce08f18f91 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:30.834037 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b8ef3-6258-40f9-8cb3-f4ce08f18f91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:30.845454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b8ef3-6258-40f9-8cb3-f4ce08f18f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:30.847195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b8ef3-6258-40f9-8cb3-f4ce08f18f91 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ca1b8ef3-6258-40f9-8cb3-f4ce08f18f91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:30.865447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:30.866332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:30.867319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:30.867319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:30.867319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:30.867319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:30.869772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 382/757] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:30.880535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b8ef3-6258-40f9-8cb3-f4ce08f18f91 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:30.923484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde322-2b52-4108-8956-50915b636c03 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:30.978314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f654f-450b-4494-804e-8a1cfed35dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:30.980761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f654f-450b-4494-804e-8a1cfed35dfc None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:30.981192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f654f-450b-4494-804e-8a1cfed35dfc None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:30.982799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f654f-450b-4494-804e-8a1cfed35dfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:30.985204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f654f-450b-4494-804e-8a1cfed35dfc None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=3dd1e02af2374224b968c2644dd0c813) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:30.988962 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f654f-450b-4494-804e-8a1cfed35dfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:30.990569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f654f-450b-4494-804e-8a1cfed35dfc None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:30.999078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:31.006877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 376/758] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:30 2022] GET /identity/v3/role_assignments?user.id=3dd1e02af2374224b968c2644dd0c813&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:31.072383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b8ef3-6258-40f9-8cb3-f4ce08f18f91 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:31.125616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cb1d4-1fe2-4ee6-b48b-e4830980409e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:31.126312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cb1d4-1fe2-4ee6-b48b-e4830980409e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:31.126598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cb1d4-1fe2-4ee6-b48b-e4830980409e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:31.126875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cb1d4-1fe2-4ee6-b48b-e4830980409e None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:31.127676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cb1d4-1fe2-4ee6-b48b-e4830980409e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:31.127995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cb1d4-1fe2-4ee6-b48b-e4830980409e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:31.129393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cb1d4-1fe2-4ee6-b48b-e4830980409e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:31.143530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 383/759] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:31.149938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f654f-450b-4494-804e-8a1cfed35dfc None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:31.183666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:31.304148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f654f-450b-4494-804e-8a1cfed35dfc None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:31.351276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70551f97-e4e3-4ff5-a77e-88f0ce759542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:31.352069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70551f97-e4e3-4ff5-a77e-88f0ce759542 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:31.352349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70551f97-e4e3-4ff5-a77e-88f0ce759542 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:31.352642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70551f97-e4e3-4ff5-a77e-88f0ce759542 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:31.363534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70551f97-e4e3-4ff5-a77e-88f0ce759542 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:31.364992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70551f97-e4e3-4ff5-a77e-88f0ce759542 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-70551f97-e4e3-4ff5-a77e-88f0ce759542 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:31.388562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:31.389607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:31.390519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 377/760] 10.222.0.47 () {56 vars in 1093 bytes} [Fri Jun 24 23:36:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:31.395961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cb1d4-1fe2-4ee6-b48b-e4830980409e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:31.526829 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cb1d4-1fe2-4ee6-b48b-e4830980409e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:31.544050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:31.573909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac9013-4396-45d5-8846-8c50e1f0763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:31.574508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac9013-4396-45d5-8846-8c50e1f0763f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:31.574787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac9013-4396-45d5-8846-8c50e1f0763f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:31.575061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac9013-4396-45d5-8846-8c50e1f0763f None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:31.575913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac9013-4396-45d5-8846-8c50e1f0763f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:31.576221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac9013-4396-45d5-8846-8c50e1f0763f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:31.578301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac9013-4396-45d5-8846-8c50e1f0763f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:31.612550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 384/761] 10.222.0.47 () {56 vars in 1133 bytes} [Fri Jun 24 23:36:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:31.617916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70551f97-e4e3-4ff5-a77e-88f0ce759542 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:31.717503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:31.756476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70551f97-e4e3-4ff5-a77e-88f0ce759542 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:31.769615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13fdfb-4701-4d28-b95f-06524a6bd56c None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:31.769904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13fdfb-4701-4d28-b95f-06524a6bd56c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:31.770178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13fdfb-4701-4d28-b95f-06524a6bd56c None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:31.771535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 385/762] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:31.781975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade100fa-55d4-479b-80d4-a79d782d5e12 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:31.782261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade100fa-55d4-479b-80d4-a79d782d5e12 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:31.782549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade100fa-55d4-479b-80d4-a79d782d5e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:31.805167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efb310-cc7b-4cba-80c3-6a286de11dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:31.805966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efb310-cc7b-4cba-80c3-6a286de11dd6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:31.806251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efb310-cc7b-4cba-80c3-6a286de11dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:31.806528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efb310-cc7b-4cba-80c3-6a286de11dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:31.817940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efb310-cc7b-4cba-80c3-6a286de11dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:31.818366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-ade100fa-55d4-479b-80d4-a79d782d5e12 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:31.819390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efb310-cc7b-4cba-80c3-6a286de11dd6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:31.819566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ade100fa-55d4-479b-80d4-a79d782d5e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-75efb310-cc7b-4cba-80c3-6a286de11dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:31.837677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:31.838536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:31.839379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:31.839379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:31.839379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:31.839379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:31.839379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 378/763] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:31.847365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efb310-cc7b-4cba-80c3-6a286de11dd6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:31.968758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 386/764] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:31.995107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efb310-cc7b-4cba-80c3-6a286de11dd6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:32.008146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade100fa-55d4-479b-80d4-a79d782d5e12 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:32.020925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:32.033118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652237d-d61e-4adc-986b-d4b7670fa2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:32.033912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652237d-d61e-4adc-986b-d4b7670fa2e7 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.034299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652237d-d61e-4adc-986b-d4b7670fa2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.034680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652237d-d61e-4adc-986b-d4b7670fa2e7 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.035508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652237d-d61e-4adc-986b-d4b7670fa2e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:32.035918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652237d-d61e-4adc-986b-d4b7670fa2e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:32.044742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652237d-d61e-4adc-986b-d4b7670fa2e7 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:32.050502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 379/765] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:32.057688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652237d-d61e-4adc-986b-d4b7670fa2e7 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:32.193552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade100fa-55d4-479b-80d4-a79d782d5e12 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:32.195975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:32.219357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652237d-d61e-4adc-986b-d4b7670fa2e7 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:32.232490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf53074-bf6a-4cb1-b4d9-29ef51fcb9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:32.233850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53074-bf6a-4cb1-b4d9-29ef51fcb9b0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.234373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53074-bf6a-4cb1-b4d9-29ef51fcb9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.234803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53074-bf6a-4cb1-b4d9-29ef51fcb9b0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.235489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf53074-bf6a-4cb1-b4d9-29ef51fcb9b0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:32.237638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf53074-bf6a-4cb1-b4d9-29ef51fcb9b0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ecf53074-bf6a-4cb1-b4d9-29ef51fcb9b0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:32.266297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:32.267290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 23:36:32.268095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:32.269918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 387/766] 10.222.0.47 () {56 vars in 1129 bytes} [Fri Jun 24 23:36:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:32.281397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2a7a9-3e9b-4bed-ade4-ea6e1feb6d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:32.282207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a7a9-3e9b-4bed-ade4-ea6e1feb6d71 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.282488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a7a9-3e9b-4bed-ade4-ea6e1feb6d71 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.282760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a7a9-3e9b-4bed-ade4-ea6e1feb6d71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.283234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f831533-4968-4643-96c0-c3b110f4eec2 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.283813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f831533-4968-4643-96c0-c3b110f4eec2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.284278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f831533-4968-4643-96c0-c3b110f4eec2 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.290772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 388/767] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:32 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:32.291460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f831533-4968-4643-96c0-c3b110f4eec2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:32.292882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2a7a9-3e9b-4bed-ade4-ea6e1feb6d71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:32.294764 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2a7a9-3e9b-4bed-ade4-ea6e1feb6d71 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-cec2a7a9-3e9b-4bed-ade4-ea6e1feb6d71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:32.312474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:32.313829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:32.314705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:32.314705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:32.314705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:32.314705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:32.314705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:32.314705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:32.314705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:32.314705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:32.321840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 380/768] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:32.323560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420be3b-c8fe-4b33-a0b5-1c95dabcf568 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.324143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420be3b-c8fe-4b33-a0b5-1c95dabcf568 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.324751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420be3b-c8fe-4b33-a0b5-1c95dabcf568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.343322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d420be3b-c8fe-4b33-a0b5-1c95dabcf568 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:32.345121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d420be3b-c8fe-4b33-a0b5-1c95dabcf568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:32.392334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:32.406581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f831533-4968-4643-96c0-c3b110f4eec2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:32.428842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 381/769] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:32.430089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420be3b-c8fe-4b33-a0b5-1c95dabcf568 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:32.451236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67458a-3db0-4187-8df8-76fbea75e069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:32.452748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67458a-3db0-4187-8df8-76fbea75e069 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.454250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67458a-3db0-4187-8df8-76fbea75e069 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.454838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67458a-3db0-4187-8df8-76fbea75e069 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.457284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67458a-3db0-4187-8df8-76fbea75e069 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:32.457846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67458a-3db0-4187-8df8-76fbea75e069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:32.459221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67458a-3db0-4187-8df8-76fbea75e069 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:32.467258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 389/770] 10.222.0.47 () {56 vars in 1124 bytes} [Fri Jun 24 23:36:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:32.469134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67458a-3db0-4187-8df8-76fbea75e069 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:32.511057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:32.575465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420be3b-c8fe-4b33-a0b5-1c95dabcf568 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:32.606422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67458a-3db0-4187-8df8-76fbea75e069 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:32.624966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88531e4-3867-45da-a87b-933620bc4d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:32.626157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88531e4-3867-45da-a87b-933620bc4d9b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.626623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88531e4-3867-45da-a87b-933620bc4d9b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.627396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88531e4-3867-45da-a87b-933620bc4d9b None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.628385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88531e4-3867-45da-a87b-933620bc4d9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:32.628900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88531e4-3867-45da-a87b-933620bc4d9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:32.630996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88531e4-3867-45da-a87b-933620bc4d9b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:32.645100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f1232-051b-4aa7-9fb1-66b0b2346f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:32.645980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f1232-051b-4aa7-9fb1-66b0b2346f43 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.646333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f1232-051b-4aa7-9fb1-66b0b2346f43 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.646737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f1232-051b-4aa7-9fb1-66b0b2346f43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.653699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 382/771] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:32.653699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88531e4-3867-45da-a87b-933620bc4d9b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:32.668406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f1232-051b-4aa7-9fb1-66b0b2346f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-f82f1232-051b-4aa7-9fb1-66b0b2346f43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:32.679774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:32.680722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:32.681541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:32.682152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 390/772] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:32.688713 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f1232-051b-4aa7-9fb1-66b0b2346f43 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:32.700766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:32.789416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88531e4-3867-45da-a87b-933620bc4d9b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:32.793957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:32.804837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f1232-051b-4aa7-9fb1-66b0b2346f43 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:32.825013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0e548-0522-469f-83a1-6e1ed55a9b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:32.825682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0e548-0522-469f-83a1-6e1ed55a9b46 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.825968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0e548-0522-469f-83a1-6e1ed55a9b46 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.826250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0e548-0522-469f-83a1-6e1ed55a9b46 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.827078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0e548-0522-469f-83a1-6e1ed55a9b46 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:32.827390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0e548-0522-469f-83a1-6e1ed55a9b46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:32.828863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0e548-0522-469f-83a1-6e1ed55a9b46 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:32.833869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 383/773] 10.222.0.47 () {56 vars in 1124 bytes} [Fri Jun 24 23:36:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:32.834494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0e548-0522-469f-83a1-6e1ed55a9b46 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:32.844399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea305ae8-9c2b-45d3-a550-31e91799a1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:32.845718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea305ae8-9c2b-45d3-a550-31e91799a1f1 None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.846212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea305ae8-9c2b-45d3-a550-31e91799a1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.846696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea305ae8-9c2b-45d3-a550-31e91799a1f1 None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/feb9bec9f1894e179e24a14b81b3f23d/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.849046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea305ae8-9c2b-45d3-a550-31e91799a1f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=feb9bec9f1894e179e24a14b81b3f23d, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:32.850674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea305ae8-9c2b-45d3-a550-31e91799a1f1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:32.865843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 391/774] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:36:32 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/feb9bec9f1894e179e24a14b81b3f23d/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:36:32.867247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4138b0e-4122-46a3-9aad-5126b36315cc None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:32.867535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4138b0e-4122-46a3-9aad-5126b36315cc None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:32.867812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4138b0e-4122-46a3-9aad-5126b36315cc None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:32.869314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 392/775] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:32.874398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4138b0e-4122-46a3-9aad-5126b36315cc None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:32.986321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:32.992822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0e548-0522-469f-83a1-6e1ed55a9b46 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.035011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade78e3-e9af-4da3-8d88-8a7c7c353e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:33.035743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade78e3-e9af-4da3-8d88-8a7c7c353e4e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.036025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade78e3-e9af-4da3-8d88-8a7c7c353e4e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.036304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade78e3-e9af-4da3-8d88-8a7c7c353e4e None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.037137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade78e3-e9af-4da3-8d88-8a7c7c353e4e None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:33.037454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade78e3-e9af-4da3-8d88-8a7c7c353e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:33.038939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade78e3-e9af-4da3-8d88-8a7c7c353e4e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:33.054859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 384/776] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:33.054859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9eb72-5b66-486d-8a21-9298bb82fb7f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.054859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9eb72-5b66-486d-8a21-9298bb82fb7f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.054859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9eb72-5b66-486d-8a21-9298bb82fb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.058162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4138b0e-4122-46a3-9aad-5126b36315cc None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.072356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c9e9eb72-5b66-486d-8a21-9298bb82fb7f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:33.073624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c9e9eb72-5b66-486d-8a21-9298bb82fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:33.105972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca8e80-6f92-42fb-a0f7-363bcd1fbde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:33.106667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca8e80-6f92-42fb-a0f7-363bcd1fbde9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.106949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca8e80-6f92-42fb-a0f7-363bcd1fbde9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.107236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca8e80-6f92-42fb-a0f7-363bcd1fbde9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.108900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca8e80-6f92-42fb-a0f7-363bcd1fbde9 None admin] RBAC: Adding query filter params (interface=public, service_id=e85632644286445fb223007b8dbb5417, region_id=RegionOne) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:33.109341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca8e80-6f92-42fb-a0f7-363bcd1fbde9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:33.110645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca8e80-6f92-42fb-a0f7-363bcd1fbde9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:33.124754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 393/777] 10.222.0.47 () {56 vars in 1251 bytes} [Fri Jun 24 23:36:32 2022] GET /identity/v3/endpoints?service_id=e85632644286445fb223007b8dbb5417&interface=public®ion_id=RegionOne => generated 194 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:33.124754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca8e80-6f92-42fb-a0f7-363bcd1fbde9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:33.147933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:33.168609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 385/778] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:33.168609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9eb72-5b66-486d-8a21-9298bb82fb7f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:33.277428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca8e80-6f92-42fb-a0f7-363bcd1fbde9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.278042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:33.289251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9eb72-5b66-486d-8a21-9298bb82fb7f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.317562 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d3695-57b6-4556-aa88-8df6c28292ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:33.318277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d3695-57b6-4556-aa88-8df6c28292ad None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.318559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d3695-57b6-4556-aa88-8df6c28292ad None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.319002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d3695-57b6-4556-aa88-8df6c28292ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.326422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a112ad-17b8-4415-a974-a944e3f8ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:33.327986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a112ad-17b8-4415-a974-a944e3f8ef61 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.328280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a112ad-17b8-4415-a974-a944e3f8ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.328565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a112ad-17b8-4415-a974-a944e3f8ef61 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.329289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a112ad-17b8-4415-a974-a944e3f8ef61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:33.330686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a112ad-17b8-4415-a974-a944e3f8ef61 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:33.331235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d3695-57b6-4556-aa88-8df6c28292ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:33.332709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d3695-57b6-4556-aa88-8df6c28292ad None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:33.336179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 386/779] 10.222.0.47 () {56 vars in 1087 bytes} [Fri Jun 24 23:36:33 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:33.336830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a112ad-17b8-4415-a974-a944e3f8ef61 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-de9d3695-57b6-4556-aa88-8df6c28292ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:33.360434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:33.361495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:33.362362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:33.362362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:33.362362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:33.362362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:33.362362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 394/780] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:33.365526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd5149-7820-4c82-9e71-e2374aaed1c5 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.365526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd5149-7820-4c82-9e71-e2374aaed1c5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.365526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd5149-7820-4c82-9e71-e2374aaed1c5 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.365857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 395/781] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:33.367719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd5149-7820-4c82-9e71-e2374aaed1c5 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:33.453384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:33.492433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a112ad-17b8-4415-a974-a944e3f8ef61 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.497311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd5149-7820-4c82-9e71-e2374aaed1c5 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.539340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea97250-018c-441b-9583-d24e058d5267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:33.540152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea97250-018c-441b-9583-d24e058d5267 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.540436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea97250-018c-441b-9583-d24e058d5267 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.540758 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea97250-018c-441b-9583-d24e058d5267 None admin] PATH_INFO: `/v3/services/network` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.541425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea97250-018c-441b-9583-d24e058d5267 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:33.542458 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a3536-3590-48d2-aadd-e95a08c01f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:33.542808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea97250-018c-441b-9583-d24e058d5267 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:33.543280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a3536-3590-48d2-aadd-e95a08c01f93 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.543560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a3536-3590-48d2-aadd-e95a08c01f93 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.543833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a3536-3590-48d2-aadd-e95a08c01f93 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.544560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a3536-3590-48d2-aadd-e95a08c01f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:33.544901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a3536-3590-48d2-aadd-e95a08c01f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:33.546392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a3536-3590-48d2-aadd-e95a08c01f93 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-6ea97250-018c-441b-9583-d24e058d5267 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:33.558953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:33.560246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 23:36:33.562402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:33.564146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 396/782] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:33.564146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ea277-3ccf-42d1-86a7-b72992252b21 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.564272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 387/783] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:33 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:33.564338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ea277-3ccf-42d1-86a7-b72992252b21 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.564338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ea277-3ccf-42d1-86a7-b72992252b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.565701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea97250-018c-441b-9583-d24e058d5267 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:33.579684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c27ea277-3ccf-42d1-86a7-b72992252b21 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:33.581058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c27ea277-3ccf-42d1-86a7-b72992252b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:33.588293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:33.656673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 397/784] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:33.658032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ea277-3ccf-42d1-86a7-b72992252b21 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:33.697345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea97250-018c-441b-9583-d24e058d5267 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.730160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:33.736550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ec90-919e-4e84-9769-49d83c63bc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:33.738458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ec90-919e-4e84-9769-49d83c63bc27 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.739364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ec90-919e-4e84-9769-49d83c63bc27 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.739839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7ec90-919e-4e84-9769-49d83c63bc27 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.748411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7ec90-919e-4e84-9769-49d83c63bc27 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:33.750194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7ec90-919e-4e84-9769-49d83c63bc27 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-55e7ec90-919e-4e84-9769-49d83c63bc27 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:33.766713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:33.767646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:33.768545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:33.771508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 388/785] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:36:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:33.773017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ec90-919e-4e84-9769-49d83c63bc27 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:33.779861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ea277-3ccf-42d1-86a7-b72992252b21 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.815789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae303db-0fec-41f7-9664-7d8b6b08c390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:33.816462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae303db-0fec-41f7-9664-7d8b6b08c390 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.816776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae303db-0fec-41f7-9664-7d8b6b08c390 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.817054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae303db-0fec-41f7-9664-7d8b6b08c390 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.817770 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae303db-0fec-41f7-9664-7d8b6b08c390 None admin] RBAC: Adding query filter params (name=network) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:33.818080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae303db-0fec-41f7-9664-7d8b6b08c390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:33.819868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae303db-0fec-41f7-9664-7d8b6b08c390 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:33.825760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 398/786] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:33.827369 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae303db-0fec-41f7-9664-7d8b6b08c390 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:33.865373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:33.892440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7ec90-919e-4e84-9769-49d83c63bc27 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.930138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e25a9c-fd9a-4333-aa04-c89c007e8902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:33.931290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e25a9c-fd9a-4333-aa04-c89c007e8902 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:33.931816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e25a9c-fd9a-4333-aa04-c89c007e8902 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:33.932382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e25a9c-fd9a-4333-aa04-c89c007e8902 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:33.946881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e25a9c-fd9a-4333-aa04-c89c007e8902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-11e25a9c-fd9a-4333-aa04-c89c007e8902 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:33.956009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:33.957124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:33.957940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:33.958559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae303db-0fec-41f7-9664-7d8b6b08c390 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:33.960836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 389/787] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:33.962679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e25a9c-fd9a-4333-aa04-c89c007e8902 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:33.985976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:34.000338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dc3b7-752e-4824-806b-8d208f6f4c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.001055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dc3b7-752e-4824-806b-8d208f6f4c55 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.001341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dc3b7-752e-4824-806b-8d208f6f4c55 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.001872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dc3b7-752e-4824-806b-8d208f6f4c55 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.002731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dc3b7-752e-4824-806b-8d208f6f4c55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:34.003043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dc3b7-752e-4824-806b-8d208f6f4c55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.004668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dc3b7-752e-4824-806b-8d208f6f4c55 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.030172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 399/788] 10.222.0.47 () {56 vars in 1149 bytes} [Fri Jun 24 23:36:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:34.031042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dc3b7-752e-4824-806b-8d208f6f4c55 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:34.095663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e25a9c-fd9a-4333-aa04-c89c007e8902 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:34.142321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6f52e-d132-4d57-8be6-2c629172fa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.143145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6f52e-d132-4d57-8be6-2c629172fa0f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.143845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6f52e-d132-4d57-8be6-2c629172fa0f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.144126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6f52e-d132-4d57-8be6-2c629172fa0f None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.145130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6f52e-d132-4d57-8be6-2c629172fa0f None admin] RBAC: Adding query filter params (type=network) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:34.145683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6f52e-d132-4d57-8be6-2c629172fa0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.147952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6f52e-d132-4d57-8be6-2c629172fa0f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.152838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dc3b7-752e-4824-806b-8d208f6f4c55 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:34.154379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 390/789] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:34.156697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6f52e-d132-4d57-8be6-2c629172fa0f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:34.160173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:34.206866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef48a1c-eec2-4fa0-a69b-42451554b3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.207691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48a1c-eec2-4fa0-a69b-42451554b3ed None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.207976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48a1c-eec2-4fa0-a69b-42451554b3ed None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.208254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef48a1c-eec2-4fa0-a69b-42451554b3ed None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.209315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef48a1c-eec2-4fa0-a69b-42451554b3ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:34.209626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef48a1c-eec2-4fa0-a69b-42451554b3ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.210981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef48a1c-eec2-4fa0-a69b-42451554b3ed None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.228616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 400/790] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:34.228616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef48a1c-eec2-4fa0-a69b-42451554b3ed None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:34.286512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6f52e-d132-4d57-8be6-2c629172fa0f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:34.295246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:34.337833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4220ee9-06ac-4505-9f01-e9ebd771ae17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.338804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220ee9-06ac-4505-9f01-e9ebd771ae17 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.339089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220ee9-06ac-4505-9f01-e9ebd771ae17 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.339376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4220ee9-06ac-4505-9f01-e9ebd771ae17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.352539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4220ee9-06ac-4505-9f01-e9ebd771ae17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.354217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4220ee9-06ac-4505-9f01-e9ebd771ae17 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.367319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef48a1c-eec2-4fa0-a69b-42451554b3ed None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-b4220ee9-06ac-4505-9f01-e9ebd771ae17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:34.372668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:34.373514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:34.374434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:34.374434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:34.374434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:34.374434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:34.376688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 391/791] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:34.377733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4220ee9-06ac-4505-9f01-e9ebd771ae17 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:34.412223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed98478-3a04-4285-a5c4-61c5a0ac0cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.412958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98478-3a04-4285-a5c4-61c5a0ac0cd9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.413379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98478-3a04-4285-a5c4-61c5a0ac0cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.413666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98478-3a04-4285-a5c4-61c5a0ac0cd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.414469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed98478-3a04-4285-a5c4-61c5a0ac0cd9 None admin] RBAC: Adding query filter params (interface=public, service_id=96be38f9cbfb44c4b96b67a8a2e47486, region_id=RegionOne) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:34.414784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed98478-3a04-4285-a5c4-61c5a0ac0cd9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.416408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed98478-3a04-4285-a5c4-61c5a0ac0cd9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.428644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 401/792] 10.222.0.47 () {56 vars in 1251 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/endpoints?service_id=96be38f9cbfb44c4b96b67a8a2e47486&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:34.428644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed98478-3a04-4285-a5c4-61c5a0ac0cd9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:34.445977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:34.497193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4220ee9-06ac-4505-9f01-e9ebd771ae17 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:34.546022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba9f8e-3c64-4486-b492-5f6616541b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.547223 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba9f8e-3c64-4486-b492-5f6616541b31 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.547508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba9f8e-3c64-4486-b492-5f6616541b31 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.547787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba9f8e-3c64-4486-b492-5f6616541b31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.552650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed98478-3a04-4285-a5c4-61c5a0ac0cd9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:34.560560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:34.569779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba9f8e-3c64-4486-b492-5f6616541b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.572257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba9f8e-3c64-4486-b492-5f6616541b31 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.593359 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cbc2e-6a0f-4253-ae6f-bbaff851ebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.594362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cbc2e-6a0f-4253-ae6f-bbaff851ebf2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-a2ba9f8e-3c64-4486-b492-5f6616541b31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:34.594521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:34.595353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:34.596233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:34.596233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:34.596233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:34.596233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:34.597216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cbc2e-6a0f-4253-ae6f-bbaff851ebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.597850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cbc2e-6a0f-4253-ae6f-bbaff851ebf2 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.598771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cbc2e-6a0f-4253-ae6f-bbaff851ebf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:34.599264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cbc2e-6a0f-4253-ae6f-bbaff851ebf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.600594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 392/793] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:34.601277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cbc2e-6a0f-4253-ae6f-bbaff851ebf2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.601443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba9f8e-3c64-4486-b492-5f6616541b31 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:34.615098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 402/794] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:34.616416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cbc2e-6a0f-4253-ae6f-bbaff851ebf2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:34.715833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba9f8e-3c64-4486-b492-5f6616541b31 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:34.722243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cbc2e-6a0f-4253-ae6f-bbaff851ebf2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:34.723998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:34.755635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218a164-ae65-4070-a60a-4db40ca978bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.756651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218a164-ae65-4070-a60a-4db40ca978bc None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.757121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218a164-ae65-4070-a60a-4db40ca978bc None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.757541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218a164-ae65-4070-a60a-4db40ca978bc None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.758558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218a164-ae65-4070-a60a-4db40ca978bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.759686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e59e-7959-4360-a3ec-93c76fe619eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.760281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218a164-ae65-4070-a60a-4db40ca978bc None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.760501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e59e-7959-4360-a3ec-93c76fe619eb None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.760881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e59e-7959-4360-a3ec-93c76fe619eb None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.761236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9e59e-7959-4360-a3ec-93c76fe619eb None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.762037 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e59e-7959-4360-a3ec-93c76fe619eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:34.762421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e59e-7959-4360-a3ec-93c76fe619eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.763837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9e59e-7959-4360-a3ec-93c76fe619eb None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.768209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 393/795] 10.222.0.47 () {56 vars in 1087 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:34.769193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218a164-ae65-4070-a60a-4db40ca978bc None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:34.788620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 403/796] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:34.796776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e59e-7959-4360-a3ec-93c76fe619eb None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:34.852426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:34.895532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218a164-ae65-4070-a60a-4db40ca978bc None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:34.943867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012edff6-fbdb-4cb3-a7a5-35dba8629879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:34.945021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012edff6-fbdb-4cb3-a7a5-35dba8629879 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:34.945454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012edff6-fbdb-4cb3-a7a5-35dba8629879 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:34.945880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012edff6-fbdb-4cb3-a7a5-35dba8629879 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:34.954979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012edff6-fbdb-4cb3-a7a5-35dba8629879 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:34.962188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012edff6-fbdb-4cb3-a7a5-35dba8629879 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-012edff6-fbdb-4cb3-a7a5-35dba8629879 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:34.980181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:34.981306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:34.982383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:34.982383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:34.982383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:34.982383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:34.982383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:34.982383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:34.982383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:34.982383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:34.985318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 394/797] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:34.992601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012edff6-fbdb-4cb3-a7a5-35dba8629879 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:35.027832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9e59e-7959-4360-a3ec-93c76fe619eb None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:35.032350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:35.078855 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231fdb5-bcc7-4c08-8045-4704b98cd21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:35.080047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231fdb5-bcc7-4c08-8045-4704b98cd21e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.080509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231fdb5-bcc7-4c08-8045-4704b98cd21e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.080986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231fdb5-bcc7-4c08-8045-4704b98cd21e None admin] PATH_INFO: `/v3/users/swift` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.089358 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231fdb5-bcc7-4c08-8045-4704b98cd21e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:35.091064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231fdb5-bcc7-4c08-8045-4704b98cd21e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-9231fdb5-bcc7-4c08-8045-4704b98cd21e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:35.107261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:35.108561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:35.109979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:35.112665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 404/798] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:35.114102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcdbe-49c9-40fc-a73c-d545e565acdc None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.118898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcdbe-49c9-40fc-a73c-d545e565acdc None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.119511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcdbe-49c9-40fc-a73c-d545e565acdc None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.121298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 405/799] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:35.122303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcdbe-49c9-40fc-a73c-d545e565acdc None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:35.128844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012edff6-fbdb-4cb3-a7a5-35dba8629879 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:35.167711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b8773-e01b-44c3-be9c-cf501379c731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:35.169025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b8773-e01b-44c3-be9c-cf501379c731 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.169401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b8773-e01b-44c3-be9c-cf501379c731 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.169734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b8773-e01b-44c3-be9c-cf501379c731 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.171817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b8773-e01b-44c3-be9c-cf501379c731 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:35.171817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b8773-e01b-44c3-be9c-cf501379c731 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:35.173781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b8773-e01b-44c3-be9c-cf501379c731 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:35.192617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 395/800] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:35.192617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369550c2-4099-4024-a495-22f9392d9160 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.192617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369550c2-4099-4024-a495-22f9392d9160 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.192617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369550c2-4099-4024-a495-22f9392d9160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.217236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-369550c2-4099-4024-a495-22f9392d9160 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:35.218816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-369550c2-4099-4024-a495-22f9392d9160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:35.234027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:35.252197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcdbe-49c9-40fc-a73c-d545e565acdc None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:35.290184 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963de6a-7cbe-4f71-b926-5f8b4e437966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:35.290993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963de6a-7cbe-4f71-b926-5f8b4e437966 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.291472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963de6a-7cbe-4f71-b926-5f8b4e437966 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.291639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963de6a-7cbe-4f71-b926-5f8b4e437966 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.292574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963de6a-7cbe-4f71-b926-5f8b4e437966 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:35.292984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963de6a-7cbe-4f71-b926-5f8b4e437966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:35.294564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963de6a-7cbe-4f71-b926-5f8b4e437966 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:35.313316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 396/801] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:35.314502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369550c2-4099-4024-a495-22f9392d9160 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:35.321849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 406/802] 10.222.0.47 () {56 vars in 1135 bytes} [Fri Jun 24 23:36:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:35.327632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963de6a-7cbe-4f71-b926-5f8b4e437966 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:35.368970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:35.438830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369550c2-4099-4024-a495-22f9392d9160 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:35.455755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963de6a-7cbe-4f71-b926-5f8b4e437966 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:35.475071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9b349-ae48-42e5-8363-474d7b97483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:35.476049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9b349-ae48-42e5-8363-474d7b97483f None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.476401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9b349-ae48-42e5-8363-474d7b97483f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.476773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9b349-ae48-42e5-8363-474d7b97483f None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/063b3fbafe0949d1b09bd9b7b0d70131/roles/147f73b08d294eae9859df32606b7d69` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.493352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9b349-ae48-42e5-8363-474d7b97483f None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=063b3fbafe0949d1b09bd9b7b0d70131, role_id=147f73b08d294eae9859df32606b7d69)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:35.493883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fbdb0-f3af-4af8-85a3-be7eaefcdf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:35.494887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9b349-ae48-42e5-8363-474d7b97483f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:35.495145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fbdb0-f3af-4af8-85a3-be7eaefcdf45 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.495603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fbdb0-f3af-4af8-85a3-be7eaefcdf45 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.496084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fbdb0-f3af-4af8-85a3-be7eaefcdf45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.507522 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fbdb0-f3af-4af8-85a3-be7eaefcdf45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:35.509394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fbdb0-f3af-4af8-85a3-be7eaefcdf45 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:35.510657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 397/803] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:36:35 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/063b3fbafe0949d1b09bd9b7b0d70131/roles/147f73b08d294eae9859df32606b7d69 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:36:35.511604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9b349-ae48-42e5-8363-474d7b97483f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:35.512922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-493fbdb0-f3af-4af8-85a3-be7eaefcdf45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:35.534220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:35.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:35.536063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:35.536063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:35.536063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:35.536063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:35.538362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 407/804] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:35.547993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fbdb0-f3af-4af8-85a3-be7eaefcdf45 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:35.595947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:35.634582 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9b349-ae48-42e5-8363-474d7b97483f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:35.677372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6d0d4-1b63-49f4-845c-7dd9e22be084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:35.678191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6d0d4-1b63-49f4-845c-7dd9e22be084 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.678472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6d0d4-1b63-49f4-845c-7dd9e22be084 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.678745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6d0d4-1b63-49f4-845c-7dd9e22be084 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.690658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fbdb0-f3af-4af8-85a3-be7eaefcdf45 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:35.691826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6d0d4-1b63-49f4-845c-7dd9e22be084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-c7b6d0d4-1b63-49f4-845c-7dd9e22be084 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:35.700084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:35.701341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:35.702300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:35.706821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 398/805] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:35.708516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f55dd3-45a3-4f17-b1a6-0d89c193ba58 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.708866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f55dd3-45a3-4f17-b1a6-0d89c193ba58 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.709189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f55dd3-45a3-4f17-b1a6-0d89c193ba58 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.711272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 399/806] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:35.712436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f55dd3-45a3-4f17-b1a6-0d89c193ba58 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:35.736082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1bb6d-31d9-41fb-9c89-e99216749a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:35.737064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1bb6d-31d9-41fb-9c89-e99216749a68 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.737524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1bb6d-31d9-41fb-9c89-e99216749a68 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.737962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1bb6d-31d9-41fb-9c89-e99216749a68 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.738919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1bb6d-31d9-41fb-9c89-e99216749a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:35.739383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1bb6d-31d9-41fb-9c89-e99216749a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:35.740996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1bb6d-31d9-41fb-9c89-e99216749a68 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:35.756009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 408/807] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:35.757691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef7fff-55aa-40a9-a47b-90b4bab44730 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.758167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef7fff-55aa-40a9-a47b-90b4bab44730 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.758626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef7fff-55aa-40a9-a47b-90b4bab44730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.777517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-01ef7fff-55aa-40a9-a47b-90b4bab44730 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:35.778987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-01ef7fff-55aa-40a9-a47b-90b4bab44730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:35.841103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f55dd3-45a3-4f17-b1a6-0d89c193ba58 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:35.848967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:35.868102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 409/808] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:35.874093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef7fff-55aa-40a9-a47b-90b4bab44730 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:35.875302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:35.882757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdeecc6-043a-4d83-8535-043cc4a6b307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:35.883697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeecc6-043a-4d83-8535-043cc4a6b307 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:35.884146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeecc6-043a-4d83-8535-043cc4a6b307 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:35.884610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdeecc6-043a-4d83-8535-043cc4a6b307 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:35.885536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdeecc6-043a-4d83-8535-043cc4a6b307 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:35.886034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdeecc6-043a-4d83-8535-043cc4a6b307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:35.887571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdeecc6-043a-4d83-8535-043cc4a6b307 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:35.898183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 400/809] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:35.904754 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdeecc6-043a-4d83-8535-043cc4a6b307 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.019755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef7fff-55aa-40a9-a47b-90b4bab44730 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.035530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdeecc6-043a-4d83-8535-043cc4a6b307 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.058035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75e35d-458d-436d-9991-7158efde01f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.058840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75e35d-458d-436d-9991-7158efde01f1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.059121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75e35d-458d-436d-9991-7158efde01f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.059393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75e35d-458d-436d-9991-7158efde01f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.067396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75e35d-458d-436d-9991-7158efde01f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.068865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75e35d-458d-436d-9991-7158efde01f1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.075268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b909fb7-14c8-4375-b32a-59dca681d9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.076297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b909fb7-14c8-4375-b32a-59dca681d9b2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.077403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b909fb7-14c8-4375-b32a-59dca681d9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.078167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b909fb7-14c8-4375-b32a-59dca681d9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-6e75e35d-458d-436d-9991-7158efde01f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:36.083644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:36.084495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:36.085708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:36.085708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:36.085708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:36.085708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:36.085708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:36.085708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:36.085708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:36.085708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:36.085708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 410/810] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:36.086177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75e35d-458d-436d-9991-7158efde01f1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.092462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b909fb7-14c8-4375-b32a-59dca681d9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.094436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b909fb7-14c8-4375-b32a-59dca681d9b2 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-0b909fb7-14c8-4375-b32a-59dca681d9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:36.115394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:36.116303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:36.117217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:36.117217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:36.117217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:36.117217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:36.119642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 401/811] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:36.120647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b909fb7-14c8-4375-b32a-59dca681d9b2 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.136068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:36.171323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:36.205586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75e35d-458d-436d-9991-7158efde01f1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.228475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b909fb7-14c8-4375-b32a-59dca681d9b2 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.251517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b534f-c0b6-4be2-ac41-a541f7ac8cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.253580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b534f-c0b6-4be2-ac41-a541f7ac8cff None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.254071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b534f-c0b6-4be2-ac41-a541f7ac8cff None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.254760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b534f-c0b6-4be2-ac41-a541f7ac8cff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.255892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b534f-c0b6-4be2-ac41-a541f7ac8cff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.257621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b534f-c0b6-4be2-ac41-a541f7ac8cff None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.267481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a35f0-d9c5-4ce4-94bc-cc3bc299828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.268317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a35f0-d9c5-4ce4-94bc-cc3bc299828e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.268781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a35f0-d9c5-4ce4-94bc-cc3bc299828e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.269206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a35f0-d9c5-4ce4-94bc-cc3bc299828e None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.270189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a35f0-d9c5-4ce4-94bc-cc3bc299828e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:36.270660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a35f0-d9c5-4ce4-94bc-cc3bc299828e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.272220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a35f0-d9c5-4ce4-94bc-cc3bc299828e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-0b4b534f-c0b6-4be2-ac41-a541f7ac8cff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:36.272502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:36.273897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 23:36:36.274915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:36.277156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 411/812] 10.222.0.47 () {56 vars in 1129 bytes} [Fri Jun 24 23:36:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:36.278050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b534f-c0b6-4be2-ac41-a541f7ac8cff None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.291743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 402/813] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:36.292910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a35f0-d9c5-4ce4-94bc-cc3bc299828e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.380049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b534f-c0b6-4be2-ac41-a541f7ac8cff None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.397148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:36.417104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e38727-5715-41e1-a67d-84e10a64edd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.417918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e38727-5715-41e1-a67d-84e10a64edd2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.418298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e38727-5715-41e1-a67d-84e10a64edd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.418745 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e38727-5715-41e1-a67d-84e10a64edd2 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.419491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e38727-5715-41e1-a67d-84e10a64edd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:36.419896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e38727-5715-41e1-a67d-84e10a64edd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.421869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e38727-5715-41e1-a67d-84e10a64edd2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.431375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a35f0-d9c5-4ce4-94bc-cc3bc299828e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.433836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 412/814] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:36.434489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:36.434804 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e38727-5715-41e1-a67d-84e10a64edd2 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.470751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22bcb7-9e7c-4bae-80c1-e99c7f6665bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.471662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22bcb7-9e7c-4bae-80c1-e99c7f6665bf None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.472079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22bcb7-9e7c-4bae-80c1-e99c7f6665bf None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.472524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22bcb7-9e7c-4bae-80c1-e99c7f6665bf None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.473401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22bcb7-9e7c-4bae-80c1-e99c7f6665bf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:36.473871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22bcb7-9e7c-4bae-80c1-e99c7f6665bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.475399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22bcb7-9e7c-4bae-80c1-e99c7f6665bf None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.480697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 403/815] 10.222.0.47 () {56 vars in 1124 bytes} [Fri Jun 24 23:36:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:36.481645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22bcb7-9e7c-4bae-80c1-e99c7f6665bf None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.538288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e38727-5715-41e1-a67d-84e10a64edd2 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.573987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64402582-a9ae-4acc-9ce3-27ff04d6a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.575021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64402582-a9ae-4acc-9ce3-27ff04d6a58f None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.575592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64402582-a9ae-4acc-9ce3-27ff04d6a58f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.575741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64402582-a9ae-4acc-9ce3-27ff04d6a58f None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/3dd1e02af2374224b968c2644dd0c813/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.578594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64402582-a9ae-4acc-9ce3-27ff04d6a58f None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=3dd1e02af2374224b968c2644dd0c813, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.580277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64402582-a9ae-4acc-9ce3-27ff04d6a58f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.589330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22bcb7-9e7c-4bae-80c1-e99c7f6665bf None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.595452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 413/816] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:36:36 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/3dd1e02af2374224b968c2644dd0c813/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:36:36.596479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64402582-a9ae-4acc-9ce3-27ff04d6a58f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.629619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5869e2-e3f6-4e64-9c56-5cbf72178446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.630710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5869e2-e3f6-4e64-9c56-5cbf72178446 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.631180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5869e2-e3f6-4e64-9c56-5cbf72178446 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.631609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5869e2-e3f6-4e64-9c56-5cbf72178446 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.636021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:36.641352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5869e2-e3f6-4e64-9c56-5cbf72178446 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.642991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5869e2-e3f6-4e64-9c56-5cbf72178446 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-7a5869e2-e3f6-4e64-9c56-5cbf72178446 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:36.659694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:36.660898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:36:36.662065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:36.664431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 404/817] 10.222.0.47 () {56 vars in 1093 bytes} [Fri Jun 24 23:36:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:36.670200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5869e2-e3f6-4e64-9c56-5cbf72178446 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.713894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:36.719876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64402582-a9ae-4acc-9ce3-27ff04d6a58f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.757561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a6902-880f-4ab0-9000-f59cf812b111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.758396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a6902-880f-4ab0-9000-f59cf812b111 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.758818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a6902-880f-4ab0-9000-f59cf812b111 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.759142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a6902-880f-4ab0-9000-f59cf812b111 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.760069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a6902-880f-4ab0-9000-f59cf812b111 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:36.760441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a6902-880f-4ab0-9000-f59cf812b111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.761989 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a6902-880f-4ab0-9000-f59cf812b111 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.769529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 414/818] 10.222.0.47 () {56 vars in 1124 bytes} [Fri Jun 24 23:36:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:36.775417 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a6902-880f-4ab0-9000-f59cf812b111 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.811494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5869e2-e3f6-4e64-9c56-5cbf72178446 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.885091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389e064-a524-4dda-979c-5662ac042ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.885923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389e064-a524-4dda-979c-5662ac042ecb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.886305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389e064-a524-4dda-979c-5662ac042ecb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.886700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389e064-a524-4dda-979c-5662ac042ecb None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.887649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389e064-a524-4dda-979c-5662ac042ecb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:36.888056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389e064-a524-4dda-979c-5662ac042ecb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.890198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389e064-a524-4dda-979c-5662ac042ecb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.913580 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 405/819] 10.222.0.47 () {56 vars in 1133 bytes} [Fri Jun 24 23:36:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:36.913996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:36.919683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389e064-a524-4dda-979c-5662ac042ecb None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:36.922824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a6902-880f-4ab0-9000-f59cf812b111 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:36.966268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b8691-9c0f-49b8-aa3b-f7c77b6f9da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:36.966944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b8691-9c0f-49b8-aa3b-f7c77b6f9da3 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:36.967412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b8691-9c0f-49b8-aa3b-f7c77b6f9da3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:36.967881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b8691-9c0f-49b8-aa3b-f7c77b6f9da3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:36.968918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b8691-9c0f-49b8-aa3b-f7c77b6f9da3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:36.970490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b8691-9c0f-49b8-aa3b-f7c77b6f9da3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:36.997012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 415/820] 10.222.0.47 () {60 vars in 1144 bytes} [Fri Jun 24 23:36:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:37.036105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:37.054234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389e064-a524-4dda-979c-5662ac042ecb None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:37.112599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa71baf-f39b-4351-aa6f-0fd807233dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:37.113474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa71baf-f39b-4351-aa6f-0fd807233dad None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:37.113862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa71baf-f39b-4351-aa6f-0fd807233dad None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:37.114239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa71baf-f39b-4351-aa6f-0fd807233dad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:37.204261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa71baf-f39b-4351-aa6f-0fd807233dad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:37.207395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929eb6-6112-43e1-bbb3-e56e485941d0 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:37.207795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929eb6-6112-43e1-bbb3-e56e485941d0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:37.208179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929eb6-6112-43e1-bbb3-e56e485941d0 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:37.209851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 416/821] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:37.211356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa71baf-f39b-4351-aa6f-0fd807233dad None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:37.218657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d005e-31b9-4e4e-8c97-88f358c13a44 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:37.219071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d005e-31b9-4e4e-8c97-88f358c13a44 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:37.219451 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d005e-31b9-4e4e-8c97-88f358c13a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:37.238031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0a1d005e-31b9-4e4e-8c97-88f358c13a44 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:37.239084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0a1d005e-31b9-4e4e-8c97-88f358c13a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:37.301729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:37.318532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 417/822] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:37.323123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 406/823] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:36:37.328640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d005e-31b9-4e4e-8c97-88f358c13a44 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:37.351609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa71baf-f39b-4351-aa6f-0fd807233dad None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:37.443497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:37.488610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d005e-31b9-4e4e-8c97-88f358c13a44 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:37.549435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1f56e-6b17-4404-a906-29d0768ca71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:37.550239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1f56e-6b17-4404-a906-29d0768ca71e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:37.550523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1f56e-6b17-4404-a906-29d0768ca71e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:37.550797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1f56e-6b17-4404-a906-29d0768ca71e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:37.554565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa71baf-f39b-4351-aa6f-0fd807233dad None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:37.573223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1f56e-6b17-4404-a906-29d0768ca71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:37.574684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1f56e-6b17-4404-a906-29d0768ca71e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-2ea1f56e-6b17-4404-a906-29d0768ca71e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:37.595817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:37.596987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:37.598027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:37.598027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:37.598027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:37.598027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:37.598027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 418/824] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:37.605820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1f56e-6b17-4404-a906-29d0768ca71e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:37.635348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c497a-1b8a-4c14-ba47-c4d04c0a5182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:37.636157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c497a-1b8a-4c14-ba47-c4d04c0a5182 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:37.636466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c497a-1b8a-4c14-ba47-c4d04c0a5182 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:37.636776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c497a-1b8a-4c14-ba47-c4d04c0a5182 None admin] PATH_INFO: `/v3/services/network` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:37.637460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c497a-1b8a-4c14-ba47-c4d04c0a5182 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:37.638862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c497a-1b8a-4c14-ba47-c4d04c0a5182 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-d81c497a-1b8a-4c14-ba47-c4d04c0a5182 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:37.660174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:37.661333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:37.662143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 407/825] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:37 2022] GET /identity/v3/services/network => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:37.672894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c497a-1b8a-4c14-ba47-c4d04c0a5182 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:37.762360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1f56e-6b17-4404-a906-29d0768ca71e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:37.790088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:37.799488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:37.814487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b4224-7540-4946-b095-87bc9e7b3fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:37.815176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b4224-7540-4946-b095-87bc9e7b3fcb None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:37.815469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b4224-7540-4946-b095-87bc9e7b3fcb None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:37.817740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b4224-7540-4946-b095-87bc9e7b3fcb None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:37.818534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b4224-7540-4946-b095-87bc9e7b3fcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:37.818860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b4224-7540-4946-b095-87bc9e7b3fcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:37.820233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b4224-7540-4946-b095-87bc9e7b3fcb None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:37.843272 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 419/826] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:37.850724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b4224-7540-4946-b095-87bc9e7b3fcb None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:37.860434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c497a-1b8a-4c14-ba47-c4d04c0a5182 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:37.912016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdc00d-b2ba-47d5-874a-89c841931dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:37.913364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdc00d-b2ba-47d5-874a-89c841931dd4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:37.913654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdc00d-b2ba-47d5-874a-89c841931dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:37.914198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdc00d-b2ba-47d5-874a-89c841931dd4 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:37.914985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdc00d-b2ba-47d5-874a-89c841931dd4 None admin] RBAC: Adding query filter params (name=network) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:37.915521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdc00d-b2ba-47d5-874a-89c841931dd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:37.918510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdc00d-b2ba-47d5-874a-89c841931dd4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:37.926022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 408/827] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:37.935275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7387c4-c503-4bee-a623-8b3e3543b775 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:37.935593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7387c4-c503-4bee-a623-8b3e3543b775 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:37.935851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7387c4-c503-4bee-a623-8b3e3543b775 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:37.939439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 409/828] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:37 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:37.940448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7387c4-c503-4bee-a623-8b3e3543b775 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:37.979463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b4224-7540-4946-b095-87bc9e7b3fcb None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:38.021317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38172e1-b53f-4dd8-9cfc-bad6b1098b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:38.022493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38172e1-b53f-4dd8-9cfc-bad6b1098b1c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:38.023294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38172e1-b53f-4dd8-9cfc-bad6b1098b1c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:38.023780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38172e1-b53f-4dd8-9cfc-bad6b1098b1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:38.032161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38172e1-b53f-4dd8-9cfc-bad6b1098b1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:38.034094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38172e1-b53f-4dd8-9cfc-bad6b1098b1c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-b38172e1-b53f-4dd8-9cfc-bad6b1098b1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:38.049798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:38.050690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:38.051566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:38.051566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:38.051566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:38.051566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:38.051566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:38.051566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:38.051566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:38.051566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:38.059482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 420/829] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:38.061235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7560d-bfc7-4363-b73d-c744ac4bd7ae None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:38.061782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7560d-bfc7-4363-b73d-c744ac4bd7ae None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:38.062304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7560d-bfc7-4363-b73d-c744ac4bd7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:38.074183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7387c4-c503-4bee-a623-8b3e3543b775 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:38.079952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c0f7560d-bfc7-4363-b73d-c744ac4bd7ae None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:38.081479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c0f7560d-bfc7-4363-b73d-c744ac4bd7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:38.088468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:38.124638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b589f7-658d-4f49-85b1-33b859a03f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:38.125484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b589f7-658d-4f49-85b1-33b859a03f5a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:38.125880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b589f7-658d-4f49-85b1-33b859a03f5a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:38.126269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b589f7-658d-4f49-85b1-33b859a03f5a None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:38.127102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b589f7-658d-4f49-85b1-33b859a03f5a None admin] RBAC: Adding query filter params (type=network) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:38.127528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b589f7-658d-4f49-85b1-33b859a03f5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:38.130125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b589f7-658d-4f49-85b1-33b859a03f5a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:38.136647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 410/830] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:38.138024 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b589f7-658d-4f49-85b1-33b859a03f5a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:38.144551 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:38.167766 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 421/831] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:38.169238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7560d-bfc7-4363-b73d-c744ac4bd7ae None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:38.264756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b589f7-658d-4f49-85b1-33b859a03f5a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:38.279061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7560d-bfc7-4363-b73d-c744ac4bd7ae None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:38.301228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2059d5-1408-421d-9c8e-2b516d1ca899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:38.301949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2059d5-1408-421d-9c8e-2b516d1ca899 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:38.302366 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2059d5-1408-421d-9c8e-2b516d1ca899 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:38.302752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2059d5-1408-421d-9c8e-2b516d1ca899 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:38.303656 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2059d5-1408-421d-9c8e-2b516d1ca899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:38.304080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2059d5-1408-421d-9c8e-2b516d1ca899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:38.305667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2059d5-1408-421d-9c8e-2b516d1ca899 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:38.317150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8488c-d5b9-4d2e-84c7-edb24832c387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:38.318025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8488c-d5b9-4d2e-84c7-edb24832c387 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:38.318451 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8488c-d5b9-4d2e-84c7-edb24832c387 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:38.318911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8488c-d5b9-4d2e-84c7-edb24832c387 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:38.320297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8488c-d5b9-4d2e-84c7-edb24832c387 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:38.321895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8488c-d5b9-4d2e-84c7-edb24832c387 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:38.324845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 411/832] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:38.325800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2059d5-1408-421d-9c8e-2b516d1ca899 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:38.343946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 422/833] 10.222.0.47 () {60 vars in 1144 bytes} [Fri Jun 24 23:36:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:38.345336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8488c-d5b9-4d2e-84c7-edb24832c387 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:38.363364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:38.399515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:38.441192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2059d5-1408-421d-9c8e-2b516d1ca899 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:38.502003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635e9ac-4b96-4ef3-8e58-64bfd8808301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:38.503086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635e9ac-4b96-4ef3-8e58-64bfd8808301 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:38.503515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635e9ac-4b96-4ef3-8e58-64bfd8808301 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:38.503968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635e9ac-4b96-4ef3-8e58-64bfd8808301 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:38.519542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8488c-d5b9-4d2e-84c7-edb24832c387 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:38.539200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635e9ac-4b96-4ef3-8e58-64bfd8808301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-5635e9ac-4b96-4ef3-8e58-64bfd8808301 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:38.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:38.548959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:36:38.549779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:38.557269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 412/834] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:36:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:38.563256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635e9ac-4b96-4ef3-8e58-64bfd8808301 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:38.581568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b36396-56c7-4921-8394-7a9397c3baaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:38.582353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36396-56c7-4921-8394-7a9397c3baaa None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:38.582632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36396-56c7-4921-8394-7a9397c3baaa None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:38.582907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b36396-56c7-4921-8394-7a9397c3baaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:38.583858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36396-56c7-4921-8394-7a9397c3baaa None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=feb9bec9f1894e179e24a14b81b3f23d) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:38.584174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36396-56c7-4921-8394-7a9397c3baaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:38.585639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b36396-56c7-4921-8394-7a9397c3baaa None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:38.600607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 423/835] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:38 2022] GET /identity/v3/role_assignments?user.id=feb9bec9f1894e179e24a14b81b3f23d&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:38.691467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:38.703346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635e9ac-4b96-4ef3-8e58-64bfd8808301 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:38.745213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7eaa06-3faa-471d-b52a-87ed96bff803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:38.745903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7eaa06-3faa-471d-b52a-87ed96bff803 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:38.746185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7eaa06-3faa-471d-b52a-87ed96bff803 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:38.746461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7eaa06-3faa-471d-b52a-87ed96bff803 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:38.747222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7eaa06-3faa-471d-b52a-87ed96bff803 None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:38.747529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7eaa06-3faa-471d-b52a-87ed96bff803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:38.749527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7eaa06-3faa-471d-b52a-87ed96bff803 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:38.758997 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 413/836] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:36:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:38.764825 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b36396-56c7-4921-8394-7a9397c3baaa None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:38.876638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b36396-56c7-4921-8394-7a9397c3baaa None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:38.878746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:38.913829 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64b769-6791-4f63-bd63-95eb374639d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:38.914678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64b769-6791-4f63-bd63-95eb374639d5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:38.914968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64b769-6791-4f63-bd63-95eb374639d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:38.915250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64b769-6791-4f63-bd63-95eb374639d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:38.927070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64b769-6791-4f63-bd63-95eb374639d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:38.928534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64b769-6791-4f63-bd63-95eb374639d5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-4a64b769-6791-4f63-bd63-95eb374639d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:38.952392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:38.953605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:38.954452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:38.954452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:38.954452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:38.954452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:38.954452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 424/837] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:38.962307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7eaa06-3faa-471d-b52a-87ed96bff803 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:39.070202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7eaa06-3faa-471d-b52a-87ed96bff803 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:39.105769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e636e-8f85-460c-87c9-bd67d085165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:39.106433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e636e-8f85-460c-87c9-bd67d085165e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.106714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e636e-8f85-460c-87c9-bd67d085165e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.106987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e636e-8f85-460c-87c9-bd67d085165e None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.107708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e636e-8f85-460c-87c9-bd67d085165e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:39.108794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e636e-8f85-460c-87c9-bd67d085165e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:39.110310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e636e-8f85-460c-87c9-bd67d085165e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:39.124205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 414/838] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:39.129596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64b769-6791-4f63-bd63-95eb374639d5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:39.190200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baafce7-6082-4f55-a04d-4910e900253f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.190491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baafce7-6082-4f55-a04d-4910e900253f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.190774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baafce7-6082-4f55-a04d-4910e900253f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.192263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 415/839] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:39.198597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0bb5a-779a-4f76-966a-a230f62a27a5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.198882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0bb5a-779a-4f76-966a-a230f62a27a5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.199159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0bb5a-779a-4f76-966a-a230f62a27a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.224402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1ef0bb5a-779a-4f76-966a-a230f62a27a5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:39.226430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1ef0bb5a-779a-4f76-966a-a230f62a27a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:39.236447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64b769-6791-4f63-bd63-95eb374639d5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:39.241301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:39.274299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7537e-7eca-48ed-a853-0258e21a7777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:39.275397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7537e-7eca-48ed-a853-0258e21a7777 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.275830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7537e-7eca-48ed-a853-0258e21a7777 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.276249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7537e-7eca-48ed-a853-0258e21a7777 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.284485 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7537e-7eca-48ed-a853-0258e21a7777 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:39.286115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7537e-7eca-48ed-a853-0258e21a7777 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ecd7537e-7eca-48ed-a853-0258e21a7777 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:39.301316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:39.302442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:39.303472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:39.311463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 416/840] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:36:39.311605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 425/841] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:36:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:39.311682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0bb5a-779a-4f76-966a-a230f62a27a5 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:39.328680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7537e-7eca-48ed-a853-0258e21a7777 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:39.346877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:39.445836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7537e-7eca-48ed-a853-0258e21a7777 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:39.446609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0bb5a-779a-4f76-966a-a230f62a27a5 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:39.485139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d3acd-23b4-479b-a7e3-c8fdc87d5416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:39.486446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d3acd-23b4-479b-a7e3-c8fdc87d5416 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.486940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d3acd-23b4-479b-a7e3-c8fdc87d5416 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.487440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d3acd-23b4-479b-a7e3-c8fdc87d5416 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.488096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12da684-5c92-40bc-92c4-2b5946ad0a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:39.488785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12da684-5c92-40bc-92c4-2b5946ad0a51 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.489069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12da684-5c92-40bc-92c4-2b5946ad0a51 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.489345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12da684-5c92-40bc-92c4-2b5946ad0a51 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.490188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12da684-5c92-40bc-92c4-2b5946ad0a51 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:39.490756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12da684-5c92-40bc-92c4-2b5946ad0a51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:39.492719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12da684-5c92-40bc-92c4-2b5946ad0a51 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:39.500497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d3acd-23b4-479b-a7e3-c8fdc87d5416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-f09d3acd-23b4-479b-a7e3-c8fdc87d5416 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:39.508760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:39.509662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:36:39.510478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:39.513018 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:39.519708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 417/842] 10.222.0.47 () {56 vars in 1149 bytes} [Fri Jun 24 23:36:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:39.519708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12da684-5c92-40bc-92c4-2b5946ad0a51 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:39.520345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 426/843] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:39.522109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d3acd-23b4-479b-a7e3-c8fdc87d5416 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:39.614840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:39.643738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d3acd-23b4-479b-a7e3-c8fdc87d5416 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:39.681055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccf9d7-cae1-48cf-a22e-a2ac52b9c2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:39.681728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccf9d7-cae1-48cf-a22e-a2ac52b9c2f4 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.682007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccf9d7-cae1-48cf-a22e-a2ac52b9c2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.682277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccf9d7-cae1-48cf-a22e-a2ac52b9c2f4 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.683025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccf9d7-cae1-48cf-a22e-a2ac52b9c2f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:39.683328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccf9d7-cae1-48cf-a22e-a2ac52b9c2f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:39.684718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ccf9d7-cae1-48cf-a22e-a2ac52b9c2f4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:39.698892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 427/844] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:39.698892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa9016-78ff-4d2c-b277-6ea40fb0473d None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.698892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa9016-78ff-4d2c-b277-6ea40fb0473d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.698892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa9016-78ff-4d2c-b277-6ea40fb0473d None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.698892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 428/845] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:39.700519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa9016-78ff-4d2c-b277-6ea40fb0473d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:39.757454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12da684-5c92-40bc-92c4-2b5946ad0a51 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:39.790715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd99fc-f0ec-4fd4-aa2e-fa05f0206c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:39.791394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd99fc-f0ec-4fd4-aa2e-fa05f0206c5b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.791677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd99fc-f0ec-4fd4-aa2e-fa05f0206c5b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.791955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd99fc-f0ec-4fd4-aa2e-fa05f0206c5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.864300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:39.867490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd99fc-f0ec-4fd4-aa2e-fa05f0206c5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:39.869175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd99fc-f0ec-4fd4-aa2e-fa05f0206c5b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:39.934598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa9016-78ff-4d2c-b277-6ea40fb0473d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:39.978523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04484f81-b764-45d5-a7a4-7faeb965b71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:39.979307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04484f81-b764-45d5-a7a4-7faeb965b71a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.979587 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04484f81-b764-45d5-a7a4-7faeb965b71a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.979858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04484f81-b764-45d5-a7a4-7faeb965b71a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.983391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:39.989030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 418/846] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:36:39.990496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b5762-3ceb-496f-b2d4-5b305d74999b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:39.990939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b5762-3ceb-496f-b2d4-5b305d74999b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:39.991378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b5762-3ceb-496f-b2d4-5b305d74999b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:39.997296 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04484f81-b764-45d5-a7a4-7faeb965b71a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:39.998862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04484f81-b764-45d5-a7a4-7faeb965b71a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.008425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3a2b5762-3ceb-496f-b2d4-5b305d74999b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:40.009809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3a2b5762-3ceb-496f-b2d4-5b305d74999b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-04484f81-b764-45d5-a7a4-7faeb965b71a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:40.016982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:40.018080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:40.021584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:40.021584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:40.021584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:40.021584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:40.021584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 429/847] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:40.021584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04484f81-b764-45d5-a7a4-7faeb965b71a None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.085090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 419/848] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:40.086179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b5762-3ceb-496f-b2d4-5b305d74999b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.127329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:40.132078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04484f81-b764-45d5-a7a4-7faeb965b71a None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.167215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298bcb9-2e78-4315-a604-cd530e9533c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.168001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298bcb9-2e78-4315-a604-cd530e9533c0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.168294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298bcb9-2e78-4315-a604-cd530e9533c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.168573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298bcb9-2e78-4315-a604-cd530e9533c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.180992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298bcb9-2e78-4315-a604-cd530e9533c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.182984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298bcb9-2e78-4315-a604-cd530e9533c0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-6298bcb9-2e78-4315-a604-cd530e9533c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:40.200462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:40.201459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:40.202919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:40.202919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:40.202919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:40.202919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:40.202919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 430/849] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:40.202919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298bcb9-2e78-4315-a604-cd530e9533c0 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.207801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b5762-3ceb-496f-b2d4-5b305d74999b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.243861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4f32-4414-421d-8194-170ed2dff75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.244552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4f32-4414-421d-8194-170ed2dff75a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.244929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4f32-4414-421d-8194-170ed2dff75a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.245275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4f32-4414-421d-8194-170ed2dff75a None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.246089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e4f32-4414-421d-8194-170ed2dff75a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:40.246470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e4f32-4414-421d-8194-170ed2dff75a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.247921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e4f32-4414-421d-8194-170ed2dff75a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.252000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:40.265419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 420/850] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:40.265419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4f32-4414-421d-8194-170ed2dff75a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.319657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298bcb9-2e78-4315-a604-cd530e9533c0 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.355628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffe781-0075-4d84-b8e6-e66c2f2f1ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.356743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffe781-0075-4d84-b8e6-e66c2f2f1ae0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.357029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffe781-0075-4d84-b8e6-e66c2f2f1ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.357306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffe781-0075-4d84-b8e6-e66c2f2f1ae0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.358749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffe781-0075-4d84-b8e6-e66c2f2f1ae0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.360736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffe781-0075-4d84-b8e6-e66c2f2f1ae0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.362498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-95ffe781-0075-4d84-b8e6-e66c2f2f1ae0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:40.373042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:40.374508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 23:36:40.375320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:40.376128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4f32-4414-421d-8194-170ed2dff75a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.376211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 431/851] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:40.376211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffe781-0075-4d84-b8e6-e66c2f2f1ae0 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.414050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c35c1-d7b8-404c-a22f-25d70a90fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.414815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c35c1-d7b8-404c-a22f-25d70a90fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.415098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c35c1-d7b8-404c-a22f-25d70a90fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.415369 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c35c1-d7b8-404c-a22f-25d70a90fdb4 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.416079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c35c1-d7b8-404c-a22f-25d70a90fdb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:40.416389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c35c1-d7b8-404c-a22f-25d70a90fdb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.417783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c35c1-d7b8-404c-a22f-25d70a90fdb4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.429709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 421/852] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:40.430507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c35c1-d7b8-404c-a22f-25d70a90fdb4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.494662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffe781-0075-4d84-b8e6-e66c2f2f1ae0 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.497338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:40.533861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4709f94-e0bc-467c-a48a-1e4d30477ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.534648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4709f94-e0bc-467c-a48a-1e4d30477ed8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.534930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4709f94-e0bc-467c-a48a-1e4d30477ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.535195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4709f94-e0bc-467c-a48a-1e4d30477ed8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.542701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4709f94-e0bc-467c-a48a-1e4d30477ed8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.544118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4709f94-e0bc-467c-a48a-1e4d30477ed8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.551044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c35c1-d7b8-404c-a22f-25d70a90fdb4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-c4709f94-e0bc-467c-a48a-1e4d30477ed8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:40.558446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:40.559536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 432/853] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:40.560452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4709f94-e0bc-467c-a48a-1e4d30477ed8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.592985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c627ab5-d34f-4e2e-b964-bd8298e2b6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.593651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c627ab5-d34f-4e2e-b964-bd8298e2b6fd None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.593934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c627ab5-d34f-4e2e-b964-bd8298e2b6fd None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.594214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c627ab5-d34f-4e2e-b964-bd8298e2b6fd None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.594917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c627ab5-d34f-4e2e-b964-bd8298e2b6fd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:40.595230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c627ab5-d34f-4e2e-b964-bd8298e2b6fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.596611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c627ab5-d34f-4e2e-b964-bd8298e2b6fd None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.604639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 422/854] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:40.604639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c627ab5-d34f-4e2e-b964-bd8298e2b6fd None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.620361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:40.671781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4709f94-e0bc-467c-a48a-1e4d30477ed8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.705763 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c627ab5-d34f-4e2e-b964-bd8298e2b6fd None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.707525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d6f76-b250-44ef-a35b-7f6c1fe21a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.708676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d6f76-b250-44ef-a35b-7f6c1fe21a6b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.708676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d6f76-b250-44ef-a35b-7f6c1fe21a6b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.708873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d6f76-b250-44ef-a35b-7f6c1fe21a6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.715641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d6f76-b250-44ef-a35b-7f6c1fe21a6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.719284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d6f76-b250-44ef-a35b-7f6c1fe21a6b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-f71d6f76-b250-44ef-a35b-7f6c1fe21a6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:40.734174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:40.735096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:40.735942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:40.735942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:40.735942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:40.735942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:40.735942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:40.735942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:40.735942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:40.735942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:40.735942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 433/855] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:40.736437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d6f76-b250-44ef-a35b-7f6c1fe21a6b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.749872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02478bc-5758-45b0-a3f8-f0be80ff6c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.750482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02478bc-5758-45b0-a3f8-f0be80ff6c1e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.750764 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02478bc-5758-45b0-a3f8-f0be80ff6c1e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.751040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02478bc-5758-45b0-a3f8-f0be80ff6c1e None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.751875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02478bc-5758-45b0-a3f8-f0be80ff6c1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:40.752184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02478bc-5758-45b0-a3f8-f0be80ff6c1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.753636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02478bc-5758-45b0-a3f8-f0be80ff6c1e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.761169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:40.778825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 423/856] 10.222.0.47 () {56 vars in 1135 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:40.778825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02478bc-5758-45b0-a3f8-f0be80ff6c1e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.842956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d6f76-b250-44ef-a35b-7f6c1fe21a6b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.849899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:40.877128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1776576-c9a0-492d-ab6f-130a8e417d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.877792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1776576-c9a0-492d-ab6f-130a8e417d12 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.878074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1776576-c9a0-492d-ab6f-130a8e417d12 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.878349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1776576-c9a0-492d-ab6f-130a8e417d12 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.878894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02478bc-5758-45b0-a3f8-f0be80ff6c1e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:40.879056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1776576-c9a0-492d-ab6f-130a8e417d12 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:40.879363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1776576-c9a0-492d-ab6f-130a8e417d12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.880751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1776576-c9a0-492d-ab6f-130a8e417d12 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.885123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 434/857] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:40.885897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1776576-c9a0-492d-ab6f-130a8e417d12 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.926147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669fafd-e6cd-4e68-81f9-66d3c867659e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:40.926816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669fafd-e6cd-4e68-81f9-66d3c867659e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.927098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669fafd-e6cd-4e68-81f9-66d3c867659e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.927370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669fafd-e6cd-4e68-81f9-66d3c867659e None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.928165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669fafd-e6cd-4e68-81f9-66d3c867659e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:40.928474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669fafd-e6cd-4e68-81f9-66d3c867659e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:40.929960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669fafd-e6cd-4e68-81f9-66d3c867659e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:40.951642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 424/858] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:40.954128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd7fbb-ba7a-4926-9490-6d06975affbf None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:40.954420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd7fbb-ba7a-4926-9490-6d06975affbf None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:40.954696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd7fbb-ba7a-4926-9490-6d06975affbf None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:40.958350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 425/859] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:40.958819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd7fbb-ba7a-4926-9490-6d06975affbf None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:40.995484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:41.009783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1776576-c9a0-492d-ab6f-130a8e417d12 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:41.054453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a84943-2fb0-4959-b33b-f7276b27125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:41.055480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a84943-2fb0-4959-b33b-f7276b27125e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:41.055774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a84943-2fb0-4959-b33b-f7276b27125e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:41.056048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a84943-2fb0-4959-b33b-f7276b27125e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:41.076611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd7fbb-ba7a-4926-9490-6d06975affbf None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:41.107954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:41.113889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aaba2c-d96d-432d-bdc6-6685ba336f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:41.114548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaba2c-d96d-432d-bdc6-6685ba336f0c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:41.114848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaba2c-d96d-432d-bdc6-6685ba336f0c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:41.115106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaba2c-d96d-432d-bdc6-6685ba336f0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:41.116262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aaba2c-d96d-432d-bdc6-6685ba336f0c None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=063b3fbafe0949d1b09bd9b7b0d70131) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:41.116556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aaba2c-d96d-432d-bdc6-6685ba336f0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:41.118015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aaba2c-d96d-432d-bdc6-6685ba336f0c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:41.128873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a84943-2fb0-4959-b33b-f7276b27125e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:41.130343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a84943-2fb0-4959-b33b-f7276b27125e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:41.144853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 426/860] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/role_assignments?user.id=063b3fbafe0949d1b09bd9b7b0d70131&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:41.144853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9d293-85ec-4781-992c-2e3ae386bfbb None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:41.144853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9d293-85ec-4781-992c-2e3ae386bfbb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:41.144853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9d293-85ec-4781-992c-2e3ae386bfbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:41.159259 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-78f9d293-85ec-4781-992c-2e3ae386bfbb None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:41.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-78f9d293-85ec-4781-992c-2e3ae386bfbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:41.247819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 427/861] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:41.274750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:41.276786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9d293-85ec-4781-992c-2e3ae386bfbb None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:41.291872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 435/862] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:40 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:36:41.299286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a84943-2fb0-4959-b33b-f7276b27125e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:41.414888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9d293-85ec-4781-992c-2e3ae386bfbb None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:41.420854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a84943-2fb0-4959-b33b-f7276b27125e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:41.447963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:41.462175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1be9d-199f-4ee4-a58b-cf08ded59a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:41.463078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1be9d-199f-4ee4-a58b-cf08ded59a73 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:41.463427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1be9d-199f-4ee4-a58b-cf08ded59a73 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:41.463746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae212c3-15d6-4628-9a6e-dca8bf9e7828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:41.463856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1be9d-199f-4ee4-a58b-cf08ded59a73 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:41.464548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1be9d-199f-4ee4-a58b-cf08ded59a73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:41.464754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae212c3-15d6-4628-9a6e-dca8bf9e7828 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:41.467930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae212c3-15d6-4628-9a6e-dca8bf9e7828 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:41.467930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae212c3-15d6-4628-9a6e-dca8bf9e7828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:41.468138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1be9d-199f-4ee4-a58b-cf08ded59a73 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:41.478235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae212c3-15d6-4628-9a6e-dca8bf9e7828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-9fc1be9d-199f-4ee4-a58b-cf08ded59a73 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:41.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:41.480497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 23:36:41.481505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:41.482467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae212c3-15d6-4628-9a6e-dca8bf9e7828 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:41.488624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 428/863] 10.222.0.47 () {56 vars in 1132 bytes} [Fri Jun 24 23:36:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:41.494646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1be9d-199f-4ee4-a58b-cf08ded59a73 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-bae212c3-15d6-4628-9a6e-dca8bf9e7828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:41.508442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:41.509580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:41.510657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:41.510657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:41.510657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:41.510657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:41.512302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 436/864] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:41.517544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae212c3-15d6-4628-9a6e-dca8bf9e7828 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:41.580624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:41.624707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1be9d-199f-4ee4-a58b-cf08ded59a73 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:41.661909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae212c3-15d6-4628-9a6e-dca8bf9e7828 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:41.675080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d4197-6178-4fba-a8a9-e3bca67f2de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:41.676027 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d4197-6178-4fba-a8a9-e3bca67f2de6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:41.676452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d4197-6178-4fba-a8a9-e3bca67f2de6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:41.676902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d4197-6178-4fba-a8a9-e3bca67f2de6 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:41.677750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d4197-6178-4fba-a8a9-e3bca67f2de6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:41.678198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d4197-6178-4fba-a8a9-e3bca67f2de6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:41.679718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d4197-6178-4fba-a8a9-e3bca67f2de6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:41.685207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 429/865] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:41.690707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d4197-6178-4fba-a8a9-e3bca67f2de6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:41.698946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc21197-63dc-4543-b84b-ada8f10cb5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:41.699788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc21197-63dc-4543-b84b-ada8f10cb5c6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:41.700233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc21197-63dc-4543-b84b-ada8f10cb5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:41.700784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc21197-63dc-4543-b84b-ada8f10cb5c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:41.701761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc21197-63dc-4543-b84b-ada8f10cb5c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:41.702210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc21197-63dc-4543-b84b-ada8f10cb5c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:41.703835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc21197-63dc-4543-b84b-ada8f10cb5c6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:41.716669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 437/866] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:41.723522 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc21197-63dc-4543-b84b-ada8f10cb5c6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:41.752308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:41.803193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d4197-6178-4fba-a8a9-e3bca67f2de6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:41.838957 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387baee-6c02-4146-8dc5-8c0b2fe05ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:41.839530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387baee-6c02-4146-8dc5-8c0b2fe05ed4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:41.839810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387baee-6c02-4146-8dc5-8c0b2fe05ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:41.840085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387baee-6c02-4146-8dc5-8c0b2fe05ed4 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:41.840815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387baee-6c02-4146-8dc5-8c0b2fe05ed4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:41.841176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387baee-6c02-4146-8dc5-8c0b2fe05ed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:41.843425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387baee-6c02-4146-8dc5-8c0b2fe05ed4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:41.848650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 430/867] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:41.862030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:41.866193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc21197-63dc-4543-b84b-ada8f10cb5c6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:41.907327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4f29f-1203-4d47-a1c3-a117966cf07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:41.908146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4f29f-1203-4d47-a1c3-a117966cf07d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:41.908425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4f29f-1203-4d47-a1c3-a117966cf07d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:41.908728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4f29f-1203-4d47-a1c3-a117966cf07d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:41.916745 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4f29f-1203-4d47-a1c3-a117966cf07d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:41.918214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4f29f-1203-4d47-a1c3-a117966cf07d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-cab4f29f-1203-4d47-a1c3-a117966cf07d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:41.934728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:41.935674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:41.936671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:41.936671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:41.936671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:41.936671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:41.936671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:41.936671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:41.936671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:41.936671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:41.936671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 438/868] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:41.947613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387baee-6c02-4146-8dc5-8c0b2fe05ed4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:42.074448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387baee-6c02-4146-8dc5-8c0b2fe05ed4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:42.094081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:42.113472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacd4ae-1640-442c-bc92-f56a332d6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:42.114149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd4ae-1640-442c-bc92-f56a332d6c20 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:42.114435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd4ae-1640-442c-bc92-f56a332d6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:42.114719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd4ae-1640-442c-bc92-f56a332d6c20 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:42.115516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacd4ae-1640-442c-bc92-f56a332d6c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:42.115831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacd4ae-1640-442c-bc92-f56a332d6c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:42.117305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacd4ae-1640-442c-bc92-f56a332d6c20 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:42.130308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 431/869] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:42.136119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4f29f-1203-4d47-a1c3-a117966cf07d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:42.246878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4f29f-1203-4d47-a1c3-a117966cf07d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:42.282197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6bac0-0bf1-49ab-81fc-c6a1632e73f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:42.282848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6bac0-0bf1-49ab-81fc-c6a1632e73f4 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:42.283127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6bac0-0bf1-49ab-81fc-c6a1632e73f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:42.283400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6bac0-0bf1-49ab-81fc-c6a1632e73f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:42.284221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6bac0-0bf1-49ab-81fc-c6a1632e73f4 None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=3dd1e02af2374224b968c2644dd0c813) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:42.284531 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6bac0-0bf1-49ab-81fc-c6a1632e73f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:42.285962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6bac0-0bf1-49ab-81fc-c6a1632e73f4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:42.292288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 439/870] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:42 2022] GET /identity/v3/role_assignments?user.id=3dd1e02af2374224b968c2644dd0c813&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:43.000271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200755c-a93f-416c-8f28-163ad2def53f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:43.000833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200755c-a93f-416c-8f28-163ad2def53f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:43.000833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200755c-a93f-416c-8f28-163ad2def53f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:43.002498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 432/871] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:43.008112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fd9a2-81f0-4ac4-b665-505768c8485d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:43.008565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fd9a2-81f0-4ac4-b665-505768c8485d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:43.008673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fd9a2-81f0-4ac4-b665-505768c8485d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:43.026967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a05fd9a2-81f0-4ac4-b665-505768c8485d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:43.028152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a05fd9a2-81f0-4ac4-b665-505768c8485d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:43.048299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:43.104697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 440/872] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:43.124574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200755c-a93f-416c-8f28-163ad2def53f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:43.235762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200755c-a93f-416c-8f28-163ad2def53f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:43.271802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357ec97-e7b8-4713-885d-fc0e514d6d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:43.272472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357ec97-e7b8-4713-885d-fc0e514d6d34 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:43.273036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357ec97-e7b8-4713-885d-fc0e514d6d34 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:43.273318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357ec97-e7b8-4713-885d-fc0e514d6d34 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:43.274058 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357ec97-e7b8-4713-885d-fc0e514d6d34 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:43.275431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357ec97-e7b8-4713-885d-fc0e514d6d34 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:43.289823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 433/873] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:36:43 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:43.464680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81cf8a-5cd0-49cf-b29c-a18b47659465 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:43.465104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81cf8a-5cd0-49cf-b29c-a18b47659465 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:43.465475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81cf8a-5cd0-49cf-b29c-a18b47659465 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:43.467072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 441/874] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:43.472514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0573-bca6-497f-9ec8-48c5f6850e9e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:43.472849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0573-bca6-497f-9ec8-48c5f6850e9e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:43.473134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb0573-bca6-497f-9ec8-48c5f6850e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:43.491301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b4fb0573-bca6-497f-9ec8-48c5f6850e9e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:43.492423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b4fb0573-bca6-497f-9ec8-48c5f6850e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:43.525400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:43.577780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 434/875] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:43.602771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81cf8a-5cd0-49cf-b29c-a18b47659465 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:43.712853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81cf8a-5cd0-49cf-b29c-a18b47659465 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:43.747623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88a467-cb0d-40b4-8425-56b233e357c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:43.748484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88a467-cb0d-40b4-8425-56b233e357c3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:43.749045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88a467-cb0d-40b4-8425-56b233e357c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:43.749364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88a467-cb0d-40b4-8425-56b233e357c3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:43.759809 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e88a467-cb0d-40b4-8425-56b233e357c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-5e88a467-cb0d-40b4-8425-56b233e357c3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:43.768786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:43.769723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:43.770569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 442/876] 10.222.0.47 () {56 vars in 1120 bytes} [Fri Jun 24 23:36:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:43.777933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb0573-bca6-497f-9ec8-48c5f6850e9e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:43.881845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb0573-bca6-497f-9ec8-48c5f6850e9e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:43.918645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a7bce-ff66-4d2d-b712-7da0b7cfab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:43.919320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a7bce-ff66-4d2d-b712-7da0b7cfab82 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:43.919604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a7bce-ff66-4d2d-b712-7da0b7cfab82 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:43.919878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a7bce-ff66-4d2d-b712-7da0b7cfab82 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:43.921352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a7bce-ff66-4d2d-b712-7da0b7cfab82 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:43.921668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a7bce-ff66-4d2d-b712-7da0b7cfab82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:43.923052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a7bce-ff66-4d2d-b712-7da0b7cfab82 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:43.931914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 435/877] 10.222.0.47 () {56 vars in 1115 bytes} [Fri Jun 24 23:36:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:43.941351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88a467-cb0d-40b4-8425-56b233e357c3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:43.959069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:44.038220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802b22a-d87c-4bc6-b1dc-6185e3c265d4 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.038711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802b22a-d87c-4bc6-b1dc-6185e3c265d4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.038711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802b22a-d87c-4bc6-b1dc-6185e3c265d4 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.040206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 436/878] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:44.045185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0a0ce-f5a7-49d1-8383-d8ea2524d1bd None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.045486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0a0ce-f5a7-49d1-8383-d8ea2524d1bd None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.045887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0a0ce-f5a7-49d1-8383-d8ea2524d1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.056747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88a467-cb0d-40b4-8425-56b233e357c3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:44.066633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-4ed0a0ce-f5a7-49d1-8383-d8ea2524d1bd None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:44.068305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4ed0a0ce-f5a7-49d1-8383-d8ea2524d1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:44.096174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f34f5c-0e88-4b8f-ae90-74e37338e586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:44.097372 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34f5c-0e88-4b8f-ae90-74e37338e586 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.097834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:44.098865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34f5c-0e88-4b8f-ae90-74e37338e586 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.099248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f34f5c-0e88-4b8f-ae90-74e37338e586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.113767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f34f5c-0e88-4b8f-ae90-74e37338e586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:44.115237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f34f5c-0e88-4b8f-ae90-74e37338e586 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-54f34f5c-0e88-4b8f-ae90-74e37338e586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:44.135002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:44.135919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:44.136827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:44.136827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:44.136827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:44.136827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:44.136827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 443/879] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:44.142711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f34f5c-0e88-4b8f-ae90-74e37338e586 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:44.167302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 437/880] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:36:44.194248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0a0ce-f5a7-49d1-8383-d8ea2524d1bd None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:44.214428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:44.250534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f34f5c-0e88-4b8f-ae90-74e37338e586 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:44.289173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad025d-65e5-4195-8782-2fef2380fdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:44.289778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad025d-65e5-4195-8782-2fef2380fdcc None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.290061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad025d-65e5-4195-8782-2fef2380fdcc None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.290341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad025d-65e5-4195-8782-2fef2380fdcc None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.291076 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad025d-65e5-4195-8782-2fef2380fdcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:44.291387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad025d-65e5-4195-8782-2fef2380fdcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:44.292782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad025d-65e5-4195-8782-2fef2380fdcc None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:44.309910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 444/881] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:44.311140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad025d-65e5-4195-8782-2fef2380fdcc None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:44.318943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0a0ce-f5a7-49d1-8383-d8ea2524d1bd None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:44.364891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a7dbf-778f-466d-abd4-4a4520bab298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:44.365863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a7dbf-778f-466d-abd4-4a4520bab298 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.366386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a7dbf-778f-466d-abd4-4a4520bab298 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.366864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a7dbf-778f-466d-abd4-4a4520bab298 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.367832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a7dbf-778f-466d-abd4-4a4520bab298 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:44.369522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a7dbf-778f-466d-abd4-4a4520bab298 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:44.393159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 438/882] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:36:44 2022] POST /identity/v3/services => generated 269 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:36:44.431746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad025d-65e5-4195-8782-2fef2380fdcc None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:44.441674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63889c2-04ca-4260-a4f9-11f7cada2006 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.441981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63889c2-04ca-4260-a4f9-11f7cada2006 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.442243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63889c2-04ca-4260-a4f9-11f7cada2006 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.443938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 439/883] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:44.449643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9aace-0728-4ced-81de-7e25efbd3866 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.449969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9aace-0728-4ced-81de-7e25efbd3866 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.450226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9aace-0728-4ced-81de-7e25efbd3866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.464718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:44.473282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-01b9aace-0728-4ced-81de-7e25efbd3866 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:44.474514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-01b9aace-0728-4ced-81de-7e25efbd3866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:44.487941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438171aa-09d5-4e89-9215-a19764928a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:44.488812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438171aa-09d5-4e89-9215-a19764928a8f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.489097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438171aa-09d5-4e89-9215-a19764928a8f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.489376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438171aa-09d5-4e89-9215-a19764928a8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.498173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:44.500114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438171aa-09d5-4e89-9215-a19764928a8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:44.501733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438171aa-09d5-4e89-9215-a19764928a8f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-438171aa-09d5-4e89-9215-a19764928a8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:44.523446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:44.524448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:44.525407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 445/884] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:36:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:44.530445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438171aa-09d5-4e89-9215-a19764928a8f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:44.580380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 440/885] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:36:44.603784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9aace-0728-4ced-81de-7e25efbd3866 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:44.672421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438171aa-09d5-4e89-9215-a19764928a8f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:44.722974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293a304-345b-420e-9bf8-edce589b1b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:44.724054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293a304-345b-420e-9bf8-edce589b1b0b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.724616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293a304-345b-420e-9bf8-edce589b1b0b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.724909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293a304-345b-420e-9bf8-edce589b1b0b None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.725765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293a304-345b-420e-9bf8-edce589b1b0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:44.726086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293a304-345b-420e-9bf8-edce589b1b0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:44.726595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9aace-0728-4ced-81de-7e25efbd3866 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:44.727525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293a304-345b-420e-9bf8-edce589b1b0b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:44.752962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 446/886] 10.222.0.47 () {56 vars in 1149 bytes} [Fri Jun 24 23:36:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:44.754852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293a304-345b-420e-9bf8-edce589b1b0b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:44.774131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15d029-b4c0-43a4-a5c2-e52e9f14a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:44.775206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15d029-b4c0-43a4-a5c2-e52e9f14a62d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.775877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15d029-b4c0-43a4-a5c2-e52e9f14a62d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.776162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15d029-b4c0-43a4-a5c2-e52e9f14a62d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.776856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15d029-b4c0-43a4-a5c2-e52e9f14a62d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:44.778237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15d029-b4c0-43a4-a5c2-e52e9f14a62d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-4a15d029-b4c0-43a4-a5c2-e52e9f14a62d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:44.793902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:44.794917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:44.795774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 441/887] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:44.802551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15d029-b4c0-43a4-a5c2-e52e9f14a62d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:44.808195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:44.819751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:44.887831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293a304-345b-420e-9bf8-edce589b1b0b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:44.922632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15d029-b4c0-43a4-a5c2-e52e9f14a62d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:44.932461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d6373-8834-407f-ac14-23b6fdc0829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:44.933288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d6373-8834-407f-ac14-23b6fdc0829f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.933725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d6373-8834-407f-ac14-23b6fdc0829f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.934100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d6373-8834-407f-ac14-23b6fdc0829f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.970541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00db3d-bfd7-401e-bb8a-0a6c34dc2024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:44.971510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00db3d-bfd7-401e-bb8a-0a6c34dc2024 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:44.971809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00db3d-bfd7-401e-bb8a-0a6c34dc2024 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:44.972094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00db3d-bfd7-401e-bb8a-0a6c34dc2024 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:44.972847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00db3d-bfd7-401e-bb8a-0a6c34dc2024 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:44.973164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00db3d-bfd7-401e-bb8a-0a6c34dc2024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:44.975080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00db3d-bfd7-401e-bb8a-0a6c34dc2024 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:44.986133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 442/888] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:36:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:44.989583 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00db3d-bfd7-401e-bb8a-0a6c34dc2024 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:45.026940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d6373-8834-407f-ac14-23b6fdc0829f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:45.028819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d6373-8834-407f-ac14-23b6fdc0829f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:45.077270 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:45.082811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:45.111213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00db3d-bfd7-401e-bb8a-0a6c34dc2024 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:45.149804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db49304-7d10-4fb4-9136-7777320d88ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:45.150264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 447/889] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:44 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 23:36:45.151053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db49304-7d10-4fb4-9136-7777320d88ad None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:45.151412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db49304-7d10-4fb4-9136-7777320d88ad None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:45.151762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db49304-7d10-4fb4-9136-7777320d88ad None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:45.152566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db49304-7d10-4fb4-9136-7777320d88ad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:45.152982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db49304-7d10-4fb4-9136-7777320d88ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:45.154467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db49304-7d10-4fb4-9136-7777320d88ad None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:45.155875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d6373-8834-407f-ac14-23b6fdc0829f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:45.164729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 443/890] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:36:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:45.273041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d6373-8834-407f-ac14-23b6fdc0829f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:45.311488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30aaf1e-ce55-40bc-ab13-6c321c2d9239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:45.311893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aaf1e-ce55-40bc-ab13-6c321c2d9239 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:45.312183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aaf1e-ce55-40bc-ab13-6c321c2d9239 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:45.312463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30aaf1e-ce55-40bc-ab13-6c321c2d9239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:45.321707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:45.323815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aaf1e-ce55-40bc-ab13-6c321c2d9239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:45.325767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30aaf1e-ce55-40bc-ab13-6c321c2d9239 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-b30aaf1e-ce55-40bc-ab13-6c321c2d9239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:45.344717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:45.345668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:45.346556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:45.346556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:45.346556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:45.346556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:45.346556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 448/891] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:45.352831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db49304-7d10-4fb4-9136-7777320d88ad None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:45.456298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db49304-7d10-4fb4-9136-7777320d88ad None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:45.491099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e610c2-ca83-4dbb-a3f0-c8a0617572c8 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:45.491727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e610c2-ca83-4dbb-a3f0-c8a0617572c8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:45.491835 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68be516-c117-4fd5-8c5a-21fa89aeda7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:45.492348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e610c2-ca83-4dbb-a3f0-c8a0617572c8 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:45.492521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68be516-c117-4fd5-8c5a-21fa89aeda7c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:45.492857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68be516-c117-4fd5-8c5a-21fa89aeda7c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:45.493137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68be516-c117-4fd5-8c5a-21fa89aeda7c None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:45.493920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68be516-c117-4fd5-8c5a-21fa89aeda7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:45.494238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68be516-c117-4fd5-8c5a-21fa89aeda7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:45.495213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 449/892] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:45.499735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68be516-c117-4fd5-8c5a-21fa89aeda7c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:45.501263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8356a-d426-4534-b46b-d909c8dcd985 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:45.501758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8356a-d426-4534-b46b-d909c8dcd985 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:45.502255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8356a-d426-4534-b46b-d909c8dcd985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:45.504337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:45.512629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 444/893] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:45.517672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68be516-c117-4fd5-8c5a-21fa89aeda7c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:45.524248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-88d8356a-d426-4534-b46b-d909c8dcd985 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:45.525775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-88d8356a-d426-4534-b46b-d909c8dcd985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:45.622087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 450/894] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:45.629118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68be516-c117-4fd5-8c5a-21fa89aeda7c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:45.652873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8356a-d426-4534-b46b-d909c8dcd985 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:45.677895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc74b58-8dc9-40cc-b5e8-9e59db41ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:45.680012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc74b58-8dc9-40cc-b5e8-9e59db41ebff None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:45.681623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc74b58-8dc9-40cc-b5e8-9e59db41ebff None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:45.682438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc74b58-8dc9-40cc-b5e8-9e59db41ebff None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:45.694185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc74b58-8dc9-40cc-b5e8-9e59db41ebff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:45.695846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc74b58-8dc9-40cc-b5e8-9e59db41ebff None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-efc74b58-8dc9-40cc-b5e8-9e59db41ebff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:45.715844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:45.716901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:45.717818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:45.717818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:45.717818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:45.717818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:45.717818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:45.717818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:45.717818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:45.717818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:45.720713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 445/895] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:45.723763 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc74b58-8dc9-40cc-b5e8-9e59db41ebff None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:45.769402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8356a-d426-4534-b46b-d909c8dcd985 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:45.793253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:45.806624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:45.808737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aec131-5c96-4b93-a3a4-0a3dae4e6c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:45.810007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aec131-5c96-4b93-a3a4-0a3dae4e6c35 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:45.810405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aec131-5c96-4b93-a3a4-0a3dae4e6c35 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:45.810874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aec131-5c96-4b93-a3a4-0a3dae4e6c35 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:45.811894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aec131-5c96-4b93-a3a4-0a3dae4e6c35 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:45.813981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aec131-5c96-4b93-a3a4-0a3dae4e6c35 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-c5aec131-5c96-4b93-a3a4-0a3dae4e6c35 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:45.829938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:45.830979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 23:36:45.831917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:45.836811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 451/896] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:45.838018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aec131-5c96-4b93-a3a4-0a3dae4e6c35 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:45.867765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc74b58-8dc9-40cc-b5e8-9e59db41ebff None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:45.909659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fa406-341d-4488-b687-44a85fef9cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:45.911082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa406-341d-4488-b687-44a85fef9cff None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:45.911566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa406-341d-4488-b687-44a85fef9cff None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:45.912315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa406-341d-4488-b687-44a85fef9cff None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:45.913488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fa406-341d-4488-b687-44a85fef9cff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:45.914000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fa406-341d-4488-b687-44a85fef9cff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:45.916187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fa406-341d-4488-b687-44a85fef9cff None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:45.932838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 446/897] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:45.940936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fa406-341d-4488-b687-44a85fef9cff None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:45.970850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aec131-5c96-4b93-a3a4-0a3dae4e6c35 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:46.013214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bd368-f864-45c8-8934-d1032e9ce0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:46.014960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bd368-f864-45c8-8934-d1032e9ce0e8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:46.015357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bd368-f864-45c8-8934-d1032e9ce0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:46.015734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bd368-f864-45c8-8934-d1032e9ce0e8 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:46.016837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bd368-f864-45c8-8934-d1032e9ce0e8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:46.017244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bd368-f864-45c8-8934-d1032e9ce0e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:46.019396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bd368-f864-45c8-8934-d1032e9ce0e8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:46.028758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 452/898] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:46.032348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bd368-f864-45c8-8934-d1032e9ce0e8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:46.068286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fa406-341d-4488-b687-44a85fef9cff None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:46.083567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:46.089101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:46.109655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77815787-1ac4-471a-9a2c-478954880157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:46.110640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77815787-1ac4-471a-9a2c-478954880157 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:46.111236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77815787-1ac4-471a-9a2c-478954880157 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:46.111718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77815787-1ac4-471a-9a2c-478954880157 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:46.112788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77815787-1ac4-471a-9a2c-478954880157 None admin] RBAC: Adding query filter params (role.id=79df14b33b3443de8396d1b92c5c1eda, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=063b3fbafe0949d1b09bd9b7b0d70131) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:46.113407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77815787-1ac4-471a-9a2c-478954880157 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:46.115147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77815787-1ac4-471a-9a2c-478954880157 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:46.129212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 447/899] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:45 2022] GET /identity/v3/role_assignments?user.id=063b3fbafe0949d1b09bd9b7b0d70131&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=79df14b33b3443de8396d1b92c5c1eda => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:46.138036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bd368-f864-45c8-8934-d1032e9ce0e8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:46.177552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7fb34-fc4b-4683-b004-a212a475e319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:46.178247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7fb34-fc4b-4683-b004-a212a475e319 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:46.178530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7fb34-fc4b-4683-b004-a212a475e319 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:46.178808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7fb34-fc4b-4683-b004-a212a475e319 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:46.179538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7fb34-fc4b-4683-b004-a212a475e319 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:46.179850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7fb34-fc4b-4683-b004-a212a475e319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:46.181291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7fb34-fc4b-4683-b004-a212a475e319 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:46.188207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 453/900] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:46.194045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77815787-1ac4-471a-9a2c-478954880157 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:46.309708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77815787-1ac4-471a-9a2c-478954880157 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:46.346954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494422f-788c-49c7-bad2-4652b77ac6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:46.347620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494422f-788c-49c7-bad2-4652b77ac6e1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:46.347902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494422f-788c-49c7-bad2-4652b77ac6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:46.348175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494422f-788c-49c7-bad2-4652b77ac6e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:46.348994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494422f-788c-49c7-bad2-4652b77ac6e1 None admin] RBAC: Adding query filter params (interface=public, service_id=6261230ef51a436fb7a6458ebd3082ca, region_id=RegionOne) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:46.349304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494422f-788c-49c7-bad2-4652b77ac6e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:46.350676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0494422f-788c-49c7-bad2-4652b77ac6e1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:46.359302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 448/901] 10.222.0.47 () {56 vars in 1251 bytes} [Fri Jun 24 23:36:46 2022] GET /identity/v3/endpoints?service_id=6261230ef51a436fb7a6458ebd3082ca&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:46.364671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7fb34-fc4b-4683-b004-a212a475e319 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:46.472292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7fb34-fc4b-4683-b004-a212a475e319 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:46.493016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:46.511106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93747433-15cf-4f00-b15a-57c88cad7c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:46.512149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93747433-15cf-4f00-b15a-57c88cad7c34 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:46.512680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93747433-15cf-4f00-b15a-57c88cad7c34 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:46.513165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93747433-15cf-4f00-b15a-57c88cad7c34 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:46.514564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93747433-15cf-4f00-b15a-57c88cad7c34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:46.516245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93747433-15cf-4f00-b15a-57c88cad7c34 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:46.522988 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 454/902] 10.222.0.47 () {56 vars in 1087 bytes} [Fri Jun 24 23:36:46 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:36:46.613599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b35c2-8140-4747-be43-6e33fffeec47 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:46.613991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b35c2-8140-4747-be43-6e33fffeec47 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:46.614165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b35c2-8140-4747-be43-6e33fffeec47 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:46.615827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 449/903] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:46.621394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8dd7-a151-43a4-a130-9d64e6181c78 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:46.622948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8dd7-a151-43a4-a130-9d64e6181c78 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:46.622948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8dd7-a151-43a4-a130-9d64e6181c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:46.641398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-e70f8dd7-a151-43a4-a130-9d64e6181c78 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:46.642978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e70f8dd7-a151-43a4-a130-9d64e6181c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:46.718505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 455/904] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:46.743985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b35c2-8140-4747-be43-6e33fffeec47 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:46.794491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:46.853995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b35c2-8140-4747-be43-6e33fffeec47 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:46.889759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43665bcb-bbe6-4085-afbe-3c94a2ef3863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:46.890532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43665bcb-bbe6-4085-afbe-3c94a2ef3863 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:46.890820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43665bcb-bbe6-4085-afbe-3c94a2ef3863 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:46.891101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43665bcb-bbe6-4085-afbe-3c94a2ef3863 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:46.891766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43665bcb-bbe6-4085-afbe-3c94a2ef3863 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:46.893233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43665bcb-bbe6-4085-afbe-3c94a2ef3863 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-43665bcb-bbe6-4085-afbe-3c94a2ef3863 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:46.907435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:46.908475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:46.909402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 450/905] 10.222.0.47 () {56 vars in 1132 bytes} [Fri Jun 24 23:36:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:46.916603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f8dd7-a151-43a4-a130-9d64e6181c78 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:47.023712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f8dd7-a151-43a4-a130-9d64e6181c78 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:47.033606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:47.059453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddbb4d-74b7-46c4-95f1-65445786f7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:47.060129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddbb4d-74b7-46c4-95f1-65445786f7b9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:47.060414 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddbb4d-74b7-46c4-95f1-65445786f7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:47.061359 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddbb4d-74b7-46c4-95f1-65445786f7b9 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:47.062097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddbb4d-74b7-46c4-95f1-65445786f7b9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:47.062409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddbb4d-74b7-46c4-95f1-65445786f7b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:47.063770 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddbb4d-74b7-46c4-95f1-65445786f7b9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:47.068917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 456/906] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:47.074379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43665bcb-bbe6-4085-afbe-3c94a2ef3863 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:47.180573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43665bcb-bbe6-4085-afbe-3c94a2ef3863 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:47.201011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:47.217988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bdb05-4f61-47f8-bf2e-e123d86cc95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:47.218738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bdb05-4f61-47f8-bf2e-e123d86cc95b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:47.219045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bdb05-4f61-47f8-bf2e-e123d86cc95b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:47.219313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bdb05-4f61-47f8-bf2e-e123d86cc95b None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:47.220363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bdb05-4f61-47f8-bf2e-e123d86cc95b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:47.220847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bdb05-4f61-47f8-bf2e-e123d86cc95b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:47.222324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bdb05-4f61-47f8-bf2e-e123d86cc95b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:47.228200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 451/907] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:47.234389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddbb4d-74b7-46c4-95f1-65445786f7b9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:47.307973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02437df9-5613-4db6-844f-a8a2d08e0007 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:47.308315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02437df9-5613-4db6-844f-a8a2d08e0007 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:47.308533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02437df9-5613-4db6-844f-a8a2d08e0007 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:47.310380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 452/908] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:47.317769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60f22c-ccfd-44b5-9997-a292a37b650d None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:47.318061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60f22c-ccfd-44b5-9997-a292a37b650d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:47.318344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60f22c-ccfd-44b5-9997-a292a37b650d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:47.340696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0d60f22c-ccfd-44b5-9997-a292a37b650d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:47.342707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0d60f22c-ccfd-44b5-9997-a292a37b650d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:47.350516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddbb4d-74b7-46c4-95f1-65445786f7b9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:47.397469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa27a3-cb22-483d-b254-7b6a91d86dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:47.398394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa27a3-cb22-483d-b254-7b6a91d86dcc None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:47.398677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa27a3-cb22-483d-b254-7b6a91d86dcc None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:47.399019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa27a3-cb22-483d-b254-7b6a91d86dcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:47.399932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa27a3-cb22-483d-b254-7b6a91d86dcc None admin] RBAC: Adding query filter params (interface=public, service_id=7355411ad73d47bfb93401521c9d32fd, region_id=RegionOne) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:47.400250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa27a3-cb22-483d-b254-7b6a91d86dcc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:47.401759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa27a3-cb22-483d-b254-7b6a91d86dcc None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:47.412240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 457/909] 10.222.0.47 () {56 vars in 1251 bytes} [Fri Jun 24 23:36:47 2022] GET /identity/v3/endpoints?service_id=7355411ad73d47bfb93401521c9d32fd&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:47.413331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa27a3-cb22-483d-b254-7b6a91d86dcc None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:47.450456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:47.520924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 453/910] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:36:47.523670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa27a3-cb22-483d-b254-7b6a91d86dcc None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:47.540103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60f22c-ccfd-44b5-9997-a292a37b650d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:47.561900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076846d-6c17-48e8-b75f-3324f00ae542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:47.562928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076846d-6c17-48e8-b75f-3324f00ae542 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:47.563549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076846d-6c17-48e8-b75f-3324f00ae542 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:47.564095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076846d-6c17-48e8-b75f-3324f00ae542 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:47.565031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076846d-6c17-48e8-b75f-3324f00ae542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:47.566640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076846d-6c17-48e8-b75f-3324f00ae542 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:47.581137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 458/911] 10.222.0.47 () {56 vars in 1087 bytes} [Fri Jun 24 23:36:47 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:36:47.736161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60f22c-ccfd-44b5-9997-a292a37b650d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:47.741084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:47.780382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f20c57-bc0d-4a50-aa73-508fe7d14192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:47.781119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20c57-bc0d-4a50-aa73-508fe7d14192 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:47.781409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20c57-bc0d-4a50-aa73-508fe7d14192 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:47.781746 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f20c57-bc0d-4a50-aa73-508fe7d14192 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:47.782504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f20c57-bc0d-4a50-aa73-508fe7d14192 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:47.783856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f20c57-bc0d-4a50-aa73-508fe7d14192 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:47.807290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 454/912] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:36:47 2022] POST /identity/v3/services => generated 241 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:48.269077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ec26-5f68-4001-b613-4aa16866f4ea None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:48.269629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ec26-5f68-4001-b613-4aa16866f4ea None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:48.269629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ec26-5f68-4001-b613-4aa16866f4ea None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:48.271240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 459/913] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:48.276901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abba994-a43b-4fdd-b23a-c36f12695b8d None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:48.277301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abba994-a43b-4fdd-b23a-c36f12695b8d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:48.277485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abba994-a43b-4fdd-b23a-c36f12695b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:48.294597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1abba994-a43b-4fdd-b23a-c36f12695b8d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:48.296063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1abba994-a43b-4fdd-b23a-c36f12695b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:48.371417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 455/914] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:48.395751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73ec26-5f68-4001-b613-4aa16866f4ea None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:48.505385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:48.510525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73ec26-5f68-4001-b613-4aa16866f4ea None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:48.546966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17b274-0b8a-47a4-93fa-cb6d3f286407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:48.547806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17b274-0b8a-47a4-93fa-cb6d3f286407 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:48.548095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17b274-0b8a-47a4-93fa-cb6d3f286407 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:48.548371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17b274-0b8a-47a4-93fa-cb6d3f286407 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:48.559291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17b274-0b8a-47a4-93fa-cb6d3f286407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-bb17b274-0b8a-47a4-93fa-cb6d3f286407 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:48.567895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:48.568854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:48.569655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 460/915] 10.222.0.47 () {56 vars in 1120 bytes} [Fri Jun 24 23:36:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:48.575534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abba994-a43b-4fdd-b23a-c36f12695b8d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:48.670760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:48.680723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abba994-a43b-4fdd-b23a-c36f12695b8d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:48.717736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628e349-b817-47b4-bbe4-c6827704a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:48.718747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628e349-b817-47b4-bbe4-c6827704a8ec None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:48.719252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628e349-b817-47b4-bbe4-c6827704a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:48.719722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628e349-b817-47b4-bbe4-c6827704a8ec None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:48.720700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628e349-b817-47b4-bbe4-c6827704a8ec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:48.721203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628e349-b817-47b4-bbe4-c6827704a8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:48.722757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628e349-b817-47b4-bbe4-c6827704a8ec None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:48.731277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 456/916] 10.222.0.47 () {56 vars in 1115 bytes} [Fri Jun 24 23:36:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:48.737475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17b274-0b8a-47a4-93fa-cb6d3f286407 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:48.842556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17b274-0b8a-47a4-93fa-cb6d3f286407 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:48.856150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793525d-be97-4261-b059-3bf0bdc4937a None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:48.856445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793525d-be97-4261-b059-3bf0bdc4937a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:48.856769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793525d-be97-4261-b059-3bf0bdc4937a None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:48.863113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 457/917] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:48.864612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f0af5-793a-44e7-8b9d-ccaf40e690de None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:48.864926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f0af5-793a-44e7-8b9d-ccaf40e690de None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:48.865266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f0af5-793a-44e7-8b9d-ccaf40e690de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:48.879100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f5e8-9c30-4e25-9c95-fdd1e7cb0f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:48.879899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f5e8-9c30-4e25-9c95-fdd1e7cb0f6b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:48.880186 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f5e8-9c30-4e25-9c95-fdd1e7cb0f6b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:48.880466 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02f5e8-9c30-4e25-9c95-fdd1e7cb0f6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:48.883916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6e6f0af5-793a-44e7-8b9d-ccaf40e690de None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:48.885157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6e6f0af5-793a-44e7-8b9d-ccaf40e690de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:48.894875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02f5e8-9c30-4e25-9c95-fdd1e7cb0f6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:48.896391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02f5e8-9c30-4e25-9c95-fdd1e7cb0f6b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-aa02f5e8-9c30-4e25-9c95-fdd1e7cb0f6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:48.914423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:48.915332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:48.916183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:48.916183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:48.916183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:48.916183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:48.916183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 461/918] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:48.921941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f5e8-9c30-4e25-9c95-fdd1e7cb0f6b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:48.943705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:48.960040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 458/919] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:36:48.985579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f0af5-793a-44e7-8b9d-ccaf40e690de None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:49.061601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02f5e8-9c30-4e25-9c95-fdd1e7cb0f6b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:49.073936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:49.095919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f0af5-793a-44e7-8b9d-ccaf40e690de None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:49.105475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356b6f9-70da-47ca-be5c-65072d252a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:49.106434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356b6f9-70da-47ca-be5c-65072d252a42 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.107228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356b6f9-70da-47ca-be5c-65072d252a42 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.107949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356b6f9-70da-47ca-be5c-65072d252a42 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.109021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356b6f9-70da-47ca-be5c-65072d252a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:49.109518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356b6f9-70da-47ca-be5c-65072d252a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:49.111586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5356b6f9-70da-47ca-be5c-65072d252a42 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:49.128167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 462/920] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:49.133838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356b6f9-70da-47ca-be5c-65072d252a42 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:49.136632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eceddb-8664-4842-87fd-c8ef3afe9c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:49.137601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eceddb-8664-4842-87fd-c8ef3afe9c67 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.138023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eceddb-8664-4842-87fd-c8ef3afe9c67 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.138456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eceddb-8664-4842-87fd-c8ef3afe9c67 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.139616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eceddb-8664-4842-87fd-c8ef3afe9c67 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:49.142120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eceddb-8664-4842-87fd-c8ef3afe9c67 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-e2eceddb-8664-4842-87fd-c8ef3afe9c67 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:49.158340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:49.159541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 23:36:49.160566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:49.163421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 459/921] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:49.168498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eceddb-8664-4842-87fd-c8ef3afe9c67 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:49.241740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:49.280681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eceddb-8664-4842-87fd-c8ef3afe9c67 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:49.288495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356b6f9-70da-47ca-be5c-65072d252a42 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:49.316889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb013317-7c27-4096-ab9a-5f24a5363178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:49.317688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb013317-7c27-4096-ab9a-5f24a5363178 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.318096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb013317-7c27-4096-ab9a-5f24a5363178 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.318806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb013317-7c27-4096-ab9a-5f24a5363178 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.319288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb013317-7c27-4096-ab9a-5f24a5363178 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:49.319701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb013317-7c27-4096-ab9a-5f24a5363178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:49.321209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb013317-7c27-4096-ab9a-5f24a5363178 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:49.331199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 460/922] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:49.333442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb013317-7c27-4096-ab9a-5f24a5363178 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:49.340024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deabdc-57c9-4b2f-96be-dc65b5ad8fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:49.341068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deabdc-57c9-4b2f-96be-dc65b5ad8fa7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.341469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deabdc-57c9-4b2f-96be-dc65b5ad8fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.342370 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42deabdc-57c9-4b2f-96be-dc65b5ad8fa7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.350327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42deabdc-57c9-4b2f-96be-dc65b5ad8fa7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:49.351213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:49.352272 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42deabdc-57c9-4b2f-96be-dc65b5ad8fa7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-42deabdc-57c9-4b2f-96be-dc65b5ad8fa7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:49.369994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:49.371182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:49.372325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:49.379004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 463/923] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:36:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:49.380107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deabdc-57c9-4b2f-96be-dc65b5ad8fa7 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:49.480953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb013317-7c27-4096-ab9a-5f24a5363178 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:49.486536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42deabdc-57c9-4b2f-96be-dc65b5ad8fa7 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:49.518752 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:49.524708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5abef-50f9-46f3-a8d7-6de686f0faf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:49.525518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5abef-50f9-46f3-a8d7-6de686f0faf5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.525851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5abef-50f9-46f3-a8d7-6de686f0faf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.526261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5abef-50f9-46f3-a8d7-6de686f0faf5 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.527317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5abef-50f9-46f3-a8d7-6de686f0faf5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:49.527731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5abef-50f9-46f3-a8d7-6de686f0faf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:49.530456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5abef-50f9-46f3-a8d7-6de686f0faf5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:49.533359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44c3f0-6728-45ae-943b-6939872aeaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:49.535090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44c3f0-6728-45ae-943b-6939872aeaf5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.535527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44c3f0-6728-45ae-943b-6939872aeaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.535962 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44c3f0-6728-45ae-943b-6939872aeaf5 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.536857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44c3f0-6728-45ae-943b-6939872aeaf5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:49.537314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44c3f0-6728-45ae-943b-6939872aeaf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:49.539152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44c3f0-6728-45ae-943b-6939872aeaf5 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:49.547945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 461/924] 10.222.0.47 () {56 vars in 1114 bytes} [Fri Jun 24 23:36:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:49.552040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 464/925] 10.222.0.47 () {56 vars in 1149 bytes} [Fri Jun 24 23:36:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:49.553160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44c3f0-6728-45ae-943b-6939872aeaf5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:49.559399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5abef-50f9-46f3-a8d7-6de686f0faf5 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:49.653516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:49.682691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44c3f0-6728-45ae-943b-6939872aeaf5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:49.690058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5abef-50f9-46f3-a8d7-6de686f0faf5 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:49.720534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4e4ea-5204-476f-b3ee-63802502571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:49.721588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4e4ea-5204-476f-b3ee-63802502571f None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.722076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4e4ea-5204-476f-b3ee-63802502571f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.722554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4e4ea-5204-476f-b3ee-63802502571f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.723510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4e4ea-5204-476f-b3ee-63802502571f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:49.725170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4e4ea-5204-476f-b3ee-63802502571f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:49.731986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e205eb-ae60-47c1-92d8-23ceffbe0594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:49.732907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e205eb-ae60-47c1-92d8-23ceffbe0594 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.733316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e205eb-ae60-47c1-92d8-23ceffbe0594 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.733649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e205eb-ae60-47c1-92d8-23ceffbe0594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.745882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e205eb-ae60-47c1-92d8-23ceffbe0594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:49.747949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e205eb-ae60-47c1-92d8-23ceffbe0594 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:49.756670 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 462/926] 10.222.0.47 () {60 vars in 1144 bytes} [Fri Jun 24 23:36:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-a1e205eb-ae60-47c1-92d8-23ceffbe0594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:49.770023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:49.770980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:49.771969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:49.771969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:49.771969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:49.771969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:49.773539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 465/927] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:49.780932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4e4ea-5204-476f-b3ee-63802502571f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:49.914191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4e4ea-5204-476f-b3ee-63802502571f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:49.926068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad207-9c5c-4d96-acbc-d1af6102f6cd None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.926374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad207-9c5c-4d96-acbc-d1af6102f6cd None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.926687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ad207-9c5c-4d96-acbc-d1af6102f6cd None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.928692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 466/928] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:49.936823 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80508cac-5b15-4036-9718-6ee0aca62ff9 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.937248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80508cac-5b15-4036-9718-6ee0aca62ff9 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.937363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80508cac-5b15-4036-9718-6ee0aca62ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.949950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:49.957761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-80508cac-5b15-4036-9718-6ee0aca62ff9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:49.959294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-80508cac-5b15-4036-9718-6ee0aca62ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:49.967385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773311c-a441-49e8-9802-3b954f03e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:49.969117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773311c-a441-49e8-9802-3b954f03e1bb None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.969404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773311c-a441-49e8-9802-3b954f03e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.969695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773311c-a441-49e8-9802-3b954f03e1bb None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.970424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773311c-a441-49e8-9802-3b954f03e1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:49.971576 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773311c-a441-49e8-9802-3b954f03e1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:49.973066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773311c-a441-49e8-9802-3b954f03e1bb None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:49.988777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 463/929] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:49.990647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64f501-965d-42b1-af9c-023cf811f94c None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:49.990647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64f501-965d-42b1-af9c-023cf811f94c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:49.990819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64f501-965d-42b1-af9c-023cf811f94c None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:49.992009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 464/930] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:49.994665 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64f501-965d-42b1-af9c-023cf811f94c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.007395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:50.041625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 467/931] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:36:50.048603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03220e69-4cb2-4dac-bab2-cf7ef09f688f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.048603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03220e69-4cb2-4dac-bab2-cf7ef09f688f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.048603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03220e69-4cb2-4dac-bab2-cf7ef09f688f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.068719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-03220e69-4cb2-4dac-bab2-cf7ef09f688f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:50.070561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-03220e69-4cb2-4dac-bab2-cf7ef09f688f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:50.116139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64f501-965d-42b1-af9c-023cf811f94c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:50.149706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 468/932] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:50.152701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ec97e-9b80-459c-9d1d-1960656be9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:50.153508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ec97e-9b80-459c-9d1d-1960656be9a6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.153783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ec97e-9b80-459c-9d1d-1960656be9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.154054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ec97e-9b80-459c-9d1d-1960656be9a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.161383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ec97e-9b80-459c-9d1d-1960656be9a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:50.161956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03220e69-4cb2-4dac-bab2-cf7ef09f688f None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.162832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ec97e-9b80-459c-9d1d-1960656be9a6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-976ec97e-9b80-459c-9d1d-1960656be9a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:50.177731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:50.178640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:50.179519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:50.179519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:50.179519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:50.179519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:50.179519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:50.179519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:50.179519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:50.179519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:50.183594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 465/933] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:50.184434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ec97e-9b80-459c-9d1d-1960656be9a6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.239200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:50.271269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03220e69-4cb2-4dac-bab2-cf7ef09f688f None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:50.273175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:50.299378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ec97e-9b80-459c-9d1d-1960656be9a6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:50.311528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e04fb-d0fd-4f5e-a789-462796d286b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:50.312326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e04fb-d0fd-4f5e-a789-462796d286b6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.312633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e04fb-d0fd-4f5e-a789-462796d286b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.312894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e04fb-d0fd-4f5e-a789-462796d286b6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.313663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e04fb-d0fd-4f5e-a789-462796d286b6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:50.314927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e04fb-d0fd-4f5e-a789-462796d286b6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-643e04fb-d0fd-4f5e-a789-462796d286b6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:50.327684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:50.328800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:50.329636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 469/934] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:50.331091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e04fb-d0fd-4f5e-a789-462796d286b6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.340824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a6551-c6ff-4e79-b96f-876f57674389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:50.341642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a6551-c6ff-4e79-b96f-876f57674389 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.342518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a6551-c6ff-4e79-b96f-876f57674389 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.342901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a6551-c6ff-4e79-b96f-876f57674389 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.343575 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a6551-c6ff-4e79-b96f-876f57674389 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:50.344976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a6551-c6ff-4e79-b96f-876f57674389 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-d31a6551-c6ff-4e79-b96f-876f57674389 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:50.357013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:50.358039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:50.359525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 466/935] 10.222.0.47 () {56 vars in 1132 bytes} [Fri Jun 24 23:36:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:50.360368 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a6551-c6ff-4e79-b96f-876f57674389 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.445309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e04fb-d0fd-4f5e-a789-462796d286b6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:50.461489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a6551-c6ff-4e79-b96f-876f57674389 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:50.471902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:50.485210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bfe86-8a4a-44c3-917a-5c39d4b1663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:50.486175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bfe86-8a4a-44c3-917a-5c39d4b1663f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.486527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bfe86-8a4a-44c3-917a-5c39d4b1663f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.486883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bfe86-8a4a-44c3-917a-5c39d4b1663f None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.487765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bfe86-8a4a-44c3-917a-5c39d4b1663f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:50.488130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bfe86-8a4a-44c3-917a-5c39d4b1663f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:50.489653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bfe86-8a4a-44c3-917a-5c39d4b1663f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:50.496257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0efac-b959-4800-aeb1-a4c845006412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:50.496948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0efac-b959-4800-aeb1-a4c845006412 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.497231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0efac-b959-4800-aeb1-a4c845006412 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.497504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0efac-b959-4800-aeb1-a4c845006412 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.498421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0efac-b959-4800-aeb1-a4c845006412 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:50.498732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0efac-b959-4800-aeb1-a4c845006412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:50.500080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0efac-b959-4800-aeb1-a4c845006412 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:50.511383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 470/936] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:50.511383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bfe86-8a4a-44c3-917a-5c39d4b1663f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.511647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 467/937] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:36:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:50.511647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0efac-b959-4800-aeb1-a4c845006412 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.542059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:50.617982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bfe86-8a4a-44c3-917a-5c39d4b1663f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:50.624523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0efac-b959-4800-aeb1-a4c845006412 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:50.660108 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a11e1-5d91-44d6-a4cf-abee602d4f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:50.660751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8d938-bb2f-41cc-9939-3962e1b354fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:50.661037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a11e1-5d91-44d6-a4cf-abee602d4f7e None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.661324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a11e1-5d91-44d6-a4cf-abee602d4f7e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.661605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a11e1-5d91-44d6-a4cf-abee602d4f7e None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/063b3fbafe0949d1b09bd9b7b0d70131/roles/79df14b33b3443de8396d1b92c5c1eda` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.661761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8d938-bb2f-41cc-9939-3962e1b354fb None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.662156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8d938-bb2f-41cc-9939-3962e1b354fb None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.662434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8d938-bb2f-41cc-9939-3962e1b354fb None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.663140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8d938-bb2f-41cc-9939-3962e1b354fb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:50.663445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8d938-bb2f-41cc-9939-3962e1b354fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:50.664060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a11e1-5d91-44d6-a4cf-abee602d4f7e None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=063b3fbafe0949d1b09bd9b7b0d70131, role_id=79df14b33b3443de8396d1b92c5c1eda)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:50.664822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8d938-bb2f-41cc-9939-3962e1b354fb None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:50.665513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a11e1-5d91-44d6-a4cf-abee602d4f7e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:50.670130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 471/938] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:50.675430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8d938-bb2f-41cc-9939-3962e1b354fb None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.679912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 468/939] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:36:50 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/063b3fbafe0949d1b09bd9b7b0d70131/roles/79df14b33b3443de8396d1b92c5c1eda => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:36:50.686155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a11e1-5d91-44d6-a4cf-abee602d4f7e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.750403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:50.784509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:50.798009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8d938-bb2f-41cc-9939-3962e1b354fb None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:50.847483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628736f-67c7-4025-bab4-1ffe7b8e4b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:50.848164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628736f-67c7-4025-bab4-1ffe7b8e4b57 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.848455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628736f-67c7-4025-bab4-1ffe7b8e4b57 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.848816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628736f-67c7-4025-bab4-1ffe7b8e4b57 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.849578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628736f-67c7-4025-bab4-1ffe7b8e4b57 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:50.849860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628736f-67c7-4025-bab4-1ffe7b8e4b57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:50.851234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628736f-67c7-4025-bab4-1ffe7b8e4b57 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:50.852842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a11e1-5d91-44d6-a4cf-abee602d4f7e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:50.856954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 472/940] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:36:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:50.863978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628736f-67c7-4025-bab4-1ffe7b8e4b57 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.895615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0941b44-49e2-46f1-865b-15fe585052e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:50.896289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0941b44-49e2-46f1-865b-15fe585052e6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:50.896573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0941b44-49e2-46f1-865b-15fe585052e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:50.896878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0941b44-49e2-46f1-865b-15fe585052e6 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:50.897847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0941b44-49e2-46f1-865b-15fe585052e6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:50.898160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0941b44-49e2-46f1-865b-15fe585052e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:50.899538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0941b44-49e2-46f1-865b-15fe585052e6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:50.905528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 469/941] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:50.912982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0941b44-49e2-46f1-865b-15fe585052e6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:50.986773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628736f-67c7-4025-bab4-1ffe7b8e4b57 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:51.036167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7381a07-dacd-414f-a0c0-34fdb3fa235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:51.037221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381a07-dacd-414f-a0c0-34fdb3fa235b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:51.037714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381a07-dacd-414f-a0c0-34fdb3fa235b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:51.038192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381a07-dacd-414f-a0c0-34fdb3fa235b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:51.039761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7381a07-dacd-414f-a0c0-34fdb3fa235b None admin] RBAC: Adding query filter params (interface=public, service_id=b5945f129e6e40cfa6cfe32674541fc9, region_id=RegionOne) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:51.040573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7381a07-dacd-414f-a0c0-34fdb3fa235b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:51.042986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7381a07-dacd-414f-a0c0-34fdb3fa235b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:51.049972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:51.054495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 473/942] 10.222.0.47 () {56 vars in 1251 bytes} [Fri Jun 24 23:36:50 2022] GET /identity/v3/endpoints?service_id=b5945f129e6e40cfa6cfe32674541fc9&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:51.055233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0941b44-49e2-46f1-865b-15fe585052e6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:51.055344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7381a07-dacd-414f-a0c0-34fdb3fa235b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:51.076539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:51.106076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2dd4f-0a16-4164-b4dd-d3a40d354b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:51.108661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2dd4f-0a16-4164-b4dd-d3a40d354b9e None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:51.109160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2dd4f-0a16-4164-b4dd-d3a40d354b9e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:51.109851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2dd4f-0a16-4164-b4dd-d3a40d354b9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:51.111179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2dd4f-0a16-4164-b4dd-d3a40d354b9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:51.112773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2dd4f-0a16-4164-b4dd-d3a40d354b9e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:51.152785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 470/943] 10.222.0.47 () {60 vars in 1144 bytes} [Fri Jun 24 23:36:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:51.188011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7381a07-dacd-414f-a0c0-34fdb3fa235b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:51.234110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fea17-b2ae-485c-908a-6dcf2e121852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:51.234799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fea17-b2ae-485c-908a-6dcf2e121852 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:51.235138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fea17-b2ae-485c-908a-6dcf2e121852 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:51.235461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fea17-b2ae-485c-908a-6dcf2e121852 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:51.236166 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fea17-b2ae-485c-908a-6dcf2e121852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:51.237734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fea17-b2ae-485c-908a-6dcf2e121852 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:51.245164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 474/944] 10.222.0.47 () {56 vars in 1087 bytes} [Fri Jun 24 23:36:51 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:36:51.896732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4cea9-09f8-4d6e-877f-c249e86e6380 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:51.897255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4cea9-09f8-4d6e-877f-c249e86e6380 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:51.897255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4cea9-09f8-4d6e-877f-c249e86e6380 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:51.899121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 471/945] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:51.904982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a83c9-02cb-4011-bbe1-42061d4af48d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:51.905431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a83c9-02cb-4011-bbe1-42061d4af48d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:51.905509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a83c9-02cb-4011-bbe1-42061d4af48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:51.923430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-155a83c9-02cb-4011-bbe1-42061d4af48d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:51.924636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-155a83c9-02cb-4011-bbe1-42061d4af48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:52.003491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 475/946] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:52.026273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4cea9-09f8-4d6e-877f-c249e86e6380 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:52.135697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4cea9-09f8-4d6e-877f-c249e86e6380 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:52.175763 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:52.176791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:52.177301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:52.177783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:52.178702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:52.180361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:52.181972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WARNING keystone.server.flask.common [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 23:36:52.201048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.sql.core [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 23:36:52.201048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 23:36:52.201048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [parameters: {'id': '01cf1543a92e41ccb125887aed7997f5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 23:36:52.201048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86604) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 01cf1543a92e41ccb125887aed7997f5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 01cf1543a92e41ccb125887aed7997f5. Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application conn.query(q) Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 23:36:52.209558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result.read() Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 23:36:52.210362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application next(self.gen) Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application yield resource Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application next(self.gen) Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise self.value Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application session.commit() Jun 24 23:36:52.211302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.session.flush() Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application compat.raise_( Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 23:36:52.212220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 23:36:52.213158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application conn.query(q) Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result.read() Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 23:36:52.213940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [parameters: {'id': '01cf1543a92e41ccb125887aed7997f5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:52.214792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:52.215560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:52.216463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:52.216463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 23:36:52.216463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 23:36:52.216463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:52.216463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:52.216463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 23:36:52.216463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 23:36:52.216463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 01cf1543a92e41ccb125887aed7997f5. Jun 24 23:36:52.216463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:52.220747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 472/947] 10.222.0.47 () {60 vars in 1140 bytes} [Fri Jun 24 23:36:52 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:52.232749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a83c9-02cb-4011-bbe1-42061d4af48d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:52.238945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:52.342808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a83c9-02cb-4011-bbe1-42061d4af48d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:52.380746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999f6a3-3033-4200-b10b-2dd75761a405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:52.381564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999f6a3-3033-4200-b10b-2dd75761a405 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:52.381846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999f6a3-3033-4200-b10b-2dd75761a405 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:52.382185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999f6a3-3033-4200-b10b-2dd75761a405 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:52.391713 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999f6a3-3033-4200-b10b-2dd75761a405 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:52.393249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999f6a3-3033-4200-b10b-2dd75761a405 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-5999f6a3-3033-4200-b10b-2dd75761a405 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:52.409457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:52.411615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:52.413810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:52.413810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:52.413810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:52.413810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:52.413810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:52.413810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:52.413810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:52.413810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:52.413810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 476/948] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:52.417909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:52.433460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:52.531780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56dd7e-e18c-4fa0-8e45-5633442ed892 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:52.567948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f13c8-3cec-43c4-9a71-1b9173dc77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:52.569068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f13c8-3cec-43c4-9a71-1b9173dc77ac None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:52.569618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f13c8-3cec-43c4-9a71-1b9173dc77ac None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:52.570098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f13c8-3cec-43c4-9a71-1b9173dc77ac None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:52.571060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f13c8-3cec-43c4-9a71-1b9173dc77ac None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:52.571579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f13c8-3cec-43c4-9a71-1b9173dc77ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:52.573456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f13c8-3cec-43c4-9a71-1b9173dc77ac None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:52.594639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 473/949] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:52.792840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c55b0-684f-4342-b6e6-a42f8334551f None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:52.793179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c55b0-684f-4342-b6e6-a42f8334551f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:52.793376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c55b0-684f-4342-b6e6-a42f8334551f None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:52.794942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 477/950] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:52.800436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae03747-6ffd-40d9-8e48-a6016e0d2b60 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:52.800764 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae03747-6ffd-40d9-8e48-a6016e0d2b60 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:52.801046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae03747-6ffd-40d9-8e48-a6016e0d2b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:52.819305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2ae03747-6ffd-40d9-8e48-a6016e0d2b60 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:52.820896 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2ae03747-6ffd-40d9-8e48-a6016e0d2b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:52.890146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:52.895387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 474/951] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:36:52.919907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c55b0-684f-4342-b6e6-a42f8334551f None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:52.998957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:53.041890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c55b0-684f-4342-b6e6-a42f8334551f None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:53.078285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c54175-e940-457d-9995-187f765bd4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:53.079420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c54175-e940-457d-9995-187f765bd4a3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.079902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c54175-e940-457d-9995-187f765bd4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.080372 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c54175-e940-457d-9995-187f765bd4a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.091889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c54175-e940-457d-9995-187f765bd4a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-f9c54175-e940-457d-9995-187f765bd4a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:36:53.100312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:53.101218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:36:53.102047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:53.108751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 478/952] 10.222.0.47 () {56 vars in 1120 bytes} [Fri Jun 24 23:36:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:53.111466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae03747-6ffd-40d9-8e48-a6016e0d2b60 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:53.216154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae03747-6ffd-40d9-8e48-a6016e0d2b60 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:53.257436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79be3d-0882-4136-b412-2e974642445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:53.258114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79be3d-0882-4136-b412-2e974642445d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.258396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79be3d-0882-4136-b412-2e974642445d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.258672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79be3d-0882-4136-b412-2e974642445d None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.259425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79be3d-0882-4136-b412-2e974642445d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:53.259728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79be3d-0882-4136-b412-2e974642445d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:53.261122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79be3d-0882-4136-b412-2e974642445d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:53.269899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 475/953] 10.222.0.47 () {56 vars in 1115 bytes} [Fri Jun 24 23:36:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:53.275715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c54175-e940-457d-9995-187f765bd4a3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:53.316493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcdadd-0573-482b-9e8c-d4870de26d4a None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.316870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcdadd-0573-482b-9e8c-d4870de26d4a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.317071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcdadd-0573-482b-9e8c-d4870de26d4a None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.318693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 476/954] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:53.323497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33094c5-e6f2-4dd4-97b4-26e9c502bdde None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.323780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33094c5-e6f2-4dd4-97b4-26e9c502bdde None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.324055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33094c5-e6f2-4dd4-97b4-26e9c502bdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.340950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b33094c5-e6f2-4dd4-97b4-26e9c502bdde None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:53.342587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b33094c5-e6f2-4dd4-97b4-26e9c502bdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:53.385235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c54175-e940-457d-9995-187f765bd4a3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:53.401519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:53.420685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 477/955] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:36:53.420685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69283c4-9bb4-4c26-b5e4-0b1ac0d81d5c None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.420685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69283c4-9bb4-4c26-b5e4-0b1ac0d81d5c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.420685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69283c4-9bb4-4c26-b5e4-0b1ac0d81d5c None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.420685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 478/956] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:53.422027 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd2b77-22a5-4d61-8745-7e0aebb23cd9 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.422717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:53.423306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd2b77-22a5-4d61-8745-7e0aebb23cd9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.423682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd2b77-22a5-4d61-8745-7e0aebb23cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.431654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a635e8-7c75-46ae-8698-e0bea4523586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:53.432460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a635e8-7c75-46ae-8698-e0bea4523586 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.432818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a635e8-7c75-46ae-8698-e0bea4523586 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.433145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a635e8-7c75-46ae-8698-e0bea4523586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.442336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9fcd2b77-22a5-4d61-8745-7e0aebb23cd9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:53.443517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9fcd2b77-22a5-4d61-8745-7e0aebb23cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:53.444826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a635e8-7c75-46ae-8698-e0bea4523586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:53.446275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a635e8-7c75-46ae-8698-e0bea4523586 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-70a635e8-7c75-46ae-8698-e0bea4523586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:53.464677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:53.465724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:53.467174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:53.467174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:53.467174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:53.467174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:53.467174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 479/957] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:53.467462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a635e8-7c75-46ae-8698-e0bea4523586 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:53.525827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 479/958] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:53.526667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd2b77-22a5-4d61-8745-7e0aebb23cd9 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:53.586375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a635e8-7c75-46ae-8698-e0bea4523586 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:53.622487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cb683-9f21-4de4-9aa4-0796cc685135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:53.623448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cb683-9f21-4de4-9aa4-0796cc685135 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.623636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cb683-9f21-4de4-9aa4-0796cc685135 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.623927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cb683-9f21-4de4-9aa4-0796cc685135 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.624532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cb683-9f21-4de4-9aa4-0796cc685135 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:53.625928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cb683-9f21-4de4-9aa4-0796cc685135 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ec5cb683-9f21-4de4-9aa4-0796cc685135 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:53.637566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:53.639147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:53.640059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 480/959] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:53.640887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cb683-9f21-4de4-9aa4-0796cc685135 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:53.649928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd2b77-22a5-4d61-8745-7e0aebb23cd9 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:53.675021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:53.688815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adcf3a-91ae-44b3-9be4-bfbd905b978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:53.689550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adcf3a-91ae-44b3-9be4-bfbd905b978a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.689769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adcf3a-91ae-44b3-9be4-bfbd905b978a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.690046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adcf3a-91ae-44b3-9be4-bfbd905b978a None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.690760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adcf3a-91ae-44b3-9be4-bfbd905b978a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:53.691071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adcf3a-91ae-44b3-9be4-bfbd905b978a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:53.691507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:53.692536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adcf3a-91ae-44b3-9be4-bfbd905b978a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:53.710604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 480/960] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:53.711404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adcf3a-91ae-44b3-9be4-bfbd905b978a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:53.750685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cb683-9f21-4de4-9aa4-0796cc685135 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:53.791102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395e101-802e-4a1e-94d1-2a5df5608b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:53.791913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395e101-802e-4a1e-94d1-2a5df5608b6d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.792191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395e101-802e-4a1e-94d1-2a5df5608b6d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.792460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395e101-802e-4a1e-94d1-2a5df5608b6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.793149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395e101-802e-4a1e-94d1-2a5df5608b6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:53.794499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395e101-802e-4a1e-94d1-2a5df5608b6d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-1395e101-802e-4a1e-94d1-2a5df5608b6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:53.808355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:53.809800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:53.810972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 481/961] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:53.811866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395e101-802e-4a1e-94d1-2a5df5608b6d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:53.826794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adcf3a-91ae-44b3-9be4-bfbd905b978a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:53.864369 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb14d-8646-4a93-8c9c-c60c3558257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:53.865069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb14d-8646-4a93-8c9c-c60c3558257b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.865352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb14d-8646-4a93-8c9c-c60c3558257b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.865624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb14d-8646-4a93-8c9c-c60c3558257b None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.866335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8eb14d-8646-4a93-8c9c-c60c3558257b None admin] RBAC: Adding query filter params (name=compute) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:53.866642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8eb14d-8646-4a93-8c9c-c60c3558257b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:53.868002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8eb14d-8646-4a93-8c9c-c60c3558257b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:53.876695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 481/962] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:53.876695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb14d-8646-4a93-8c9c-c60c3558257b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:53.922139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:53.927267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395e101-802e-4a1e-94d1-2a5df5608b6d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:53.940230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:53.970642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6b009-e242-465e-8ab9-2f396bacdd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:53.971627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6b009-e242-465e-8ab9-2f396bacdd75 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:53.971758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6b009-e242-465e-8ab9-2f396bacdd75 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:53.971997 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6b009-e242-465e-8ab9-2f396bacdd75 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:53.985660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6b009-e242-465e-8ab9-2f396bacdd75 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:53.985935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb14d-8646-4a93-8c9c-c60c3558257b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:53.987119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6b009-e242-465e-8ab9-2f396bacdd75 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-92b6b009-e242-465e-8ab9-2f396bacdd75 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:54.002991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:54.003913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:54.004881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 482/963] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:36:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:54.005483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6b009-e242-465e-8ab9-2f396bacdd75 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:54.032114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80d4eb-79a1-457d-9aab-212e705a947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:54.032818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80d4eb-79a1-457d-9aab-212e705a947e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.033093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80d4eb-79a1-457d-9aab-212e705a947e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.033384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80d4eb-79a1-457d-9aab-212e705a947e None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.034082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80d4eb-79a1-457d-9aab-212e705a947e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:54.034421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80d4eb-79a1-457d-9aab-212e705a947e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:54.035761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80d4eb-79a1-457d-9aab-212e705a947e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:54.040840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 482/964] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:36:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:54.041512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80d4eb-79a1-457d-9aab-212e705a947e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:54.122352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6b009-e242-465e-8ab9-2f396bacdd75 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:54.145864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80d4eb-79a1-457d-9aab-212e705a947e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:54.157806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:54.161363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75b054-faac-4982-a5d0-00fd21df1353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:54.162036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75b054-faac-4982-a5d0-00fd21df1353 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.162324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75b054-faac-4982-a5d0-00fd21df1353 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.162604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75b054-faac-4982-a5d0-00fd21df1353 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.163312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75b054-faac-4982-a5d0-00fd21df1353 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:54.163642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75b054-faac-4982-a5d0-00fd21df1353 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:54.165020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75b054-faac-4982-a5d0-00fd21df1353 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:54.180630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 483/965] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:54.180630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75b054-faac-4982-a5d0-00fd21df1353 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:54.196030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cd69b-c013-4e71-9028-a0d9cf6ed62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:54.196860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cd69b-c013-4e71-9028-a0d9cf6ed62e None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.197218 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cd69b-c013-4e71-9028-a0d9cf6ed62e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.197589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cd69b-c013-4e71-9028-a0d9cf6ed62e None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.198526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cd69b-c013-4e71-9028-a0d9cf6ed62e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:54.198907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cd69b-c013-4e71-9028-a0d9cf6ed62e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:54.200538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cd69b-c013-4e71-9028-a0d9cf6ed62e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:54.218442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:54.221275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 483/966] 10.222.0.47 () {56 vars in 1149 bytes} [Fri Jun 24 23:36:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:54.227448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cd69b-c013-4e71-9028-a0d9cf6ed62e None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:54.322456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75b054-faac-4982-a5d0-00fd21df1353 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:54.355893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cd69b-c013-4e71-9028-a0d9cf6ed62e None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:54.372486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf962c-add3-4640-b829-b4a73fb12918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:54.373841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf962c-add3-4640-b829-b4a73fb12918 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.374128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf962c-add3-4640-b829-b4a73fb12918 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.374411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf962c-add3-4640-b829-b4a73fb12918 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.375138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf962c-add3-4640-b829-b4a73fb12918 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:54.375453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf962c-add3-4640-b829-b4a73fb12918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:54.377005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf962c-add3-4640-b829-b4a73fb12918 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:54.383938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 484/967] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:36:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:54.391544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf962c-add3-4640-b829-b4a73fb12918 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:54.398453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebf68f-394d-4dd5-95f3-4c8c254582f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:54.399050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebf68f-394d-4dd5-95f3-4c8c254582f6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.399331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebf68f-394d-4dd5-95f3-4c8c254582f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.399605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebf68f-394d-4dd5-95f3-4c8c254582f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.439441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:54.480212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebf68f-394d-4dd5-95f3-4c8c254582f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:54.482613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebf68f-394d-4dd5-95f3-4c8c254582f6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:54.512335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:54.520796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf962c-add3-4640-b829-b4a73fb12918 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:54.559716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94719ada-876e-4076-90a9-233c0423e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:54.561173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94719ada-876e-4076-90a9-233c0423e939 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.561560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94719ada-876e-4076-90a9-233c0423e939 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.561985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94719ada-876e-4076-90a9-233c0423e939 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.562836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94719ada-876e-4076-90a9-233c0423e939 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:54.564471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94719ada-876e-4076-90a9-233c0423e939 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:54.593001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 485/968] 10.222.0.47 () {60 vars in 1144 bytes} [Fri Jun 24 23:36:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:36:54.608932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 484/969] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:36:54 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:36:54.614675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94719ada-876e-4076-90a9-233c0423e939 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:54.745677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94719ada-876e-4076-90a9-233c0423e939 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:54.782980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867d064-5f28-47a0-8320-bee235acc783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:54.783784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867d064-5f28-47a0-8320-bee235acc783 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.784127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867d064-5f28-47a0-8320-bee235acc783 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.784411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867d064-5f28-47a0-8320-bee235acc783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.795996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867d064-5f28-47a0-8320-bee235acc783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:54.797619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867d064-5f28-47a0-8320-bee235acc783 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:54.806291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e421d65-6997-4461-a221-d8459dee115c None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.806586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e421d65-6997-4461-a221-d8459dee115c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.806866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e421d65-6997-4461-a221-d8459dee115c None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.806965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:54.809515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 485/970] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:36:54.814850 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c3dcf-8b01-443c-845e-8b9cfb9f79db None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.815149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c3dcf-8b01-443c-845e-8b9cfb9f79db None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.815436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c3dcf-8b01-443c-845e-8b9cfb9f79db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-4867d064-5f28-47a0-8320-bee235acc783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:36:54.819547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:54.820402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:54.821322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:36:54.821322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:36:54.821322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:36:54.821322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:54.823866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 486/971] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:54.830055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867d064-5f28-47a0-8320-bee235acc783 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:54.833133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3c5c3dcf-8b01-443c-845e-8b9cfb9f79db None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:54.834830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3c5c3dcf-8b01-443c-845e-8b9cfb9f79db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:54.928450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:54.940687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 486/972] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:36:54.953821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867d064-5f28-47a0-8320-bee235acc783 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:54.963698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c3dcf-8b01-443c-845e-8b9cfb9f79db None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:54.995487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5538598-b7ab-47af-b0c8-814e8cb6f7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:54.996473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538598-b7ab-47af-b0c8-814e8cb6f7c7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:54.997064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538598-b7ab-47af-b0c8-814e8cb6f7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:54.997506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538598-b7ab-47af-b0c8-814e8cb6f7c7 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:54.999004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5538598-b7ab-47af-b0c8-814e8cb6f7c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:54.999479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5538598-b7ab-47af-b0c8-814e8cb6f7c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:55.001208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5538598-b7ab-47af-b0c8-814e8cb6f7c7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:55.015311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 487/973] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:55.020752 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5538598-b7ab-47af-b0c8-814e8cb6f7c7 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:55.090248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:55.112732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c3dcf-8b01-443c-845e-8b9cfb9f79db None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:55.142747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5538598-b7ab-47af-b0c8-814e8cb6f7c7 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:55.150623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:55.151882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:55.152171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:55.152452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:55.153217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:55.154687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:55.170387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.sql.core [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 23:36:55.170387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 23:36:55.170387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [parameters: {'user_id': '2bd2f50677b047beacc41e616ea12201', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 23:36:55.170387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86604) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application conn.query(q) Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result.read() Jun 24 23:36:55.178022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 23:36:55.178922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application next(self.gen) Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application yield resource Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application next(self.gen) Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise self.value Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application session.commit() Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 23:36:55.179875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.session.flush() Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application compat.raise_( Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 23:36:55.181329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 23:36:55.182184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application conn.query(q) Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application result.read() Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 23:36:55.184268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [parameters: {'user_id': '2bd2f50677b047beacc41e616ea12201', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:55.186489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 23:36:55.188619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 23:36:55.191766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 23:36:55.191766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 23:36:55.191766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 23:36:55.191766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 23:36:55.191766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 23:36:55.191766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:55.191766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 487/974] 10.222.0.47 () {60 vars in 1131 bytes} [Fri Jun 24 23:36:54 2022] POST /identity/v3/users => generated 160 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:55.191766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:55.209547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91796d6a-cffa-491e-bd11-9e8d317f85ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:55.211197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91796d6a-cffa-491e-bd11-9e8d317f85ad None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:55.211989 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91796d6a-cffa-491e-bd11-9e8d317f85ad None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:55.212481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91796d6a-cffa-491e-bd11-9e8d317f85ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:55.223275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91796d6a-cffa-491e-bd11-9e8d317f85ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:55.226206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91796d6a-cffa-491e-bd11-9e8d317f85ad None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-91796d6a-cffa-491e-bd11-9e8d317f85ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:36:55.244310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:55.245258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:55.246379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:36:55.246379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:36:55.246379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:36:55.246379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:36:55.246379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:36:55.246379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:36:55.246379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:36:55.246379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:55.249002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 488/975] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:36:55.254260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91796d6a-cffa-491e-bd11-9e8d317f85ad None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:55.259959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:55.318778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715106d-ba19-4760-8533-075ee96ae1d4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:55.360356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b38c10-6ac4-4658-ba11-bdc2349e13e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:55.361573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b38c10-6ac4-4658-ba11-bdc2349e13e8 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:55.362121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b38c10-6ac4-4658-ba11-bdc2349e13e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:55.362615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b38c10-6ac4-4658-ba11-bdc2349e13e8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:55.371343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b38c10-6ac4-4658-ba11-bdc2349e13e8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:55.373952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b38c10-6ac4-4658-ba11-bdc2349e13e8 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:55.387879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91796d6a-cffa-491e-bd11-9e8d317f85ad None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-58b38c10-6ac4-4658-ba11-bdc2349e13e8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:36:55.389722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:55.390624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 23:36:55.391530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:55.398845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 488/976] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:36:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:55.399957 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b38c10-6ac4-4658-ba11-bdc2349e13e8 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:55.410855 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:55.428166 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e58cb0-4912-4cff-bdcb-7db47b9ba2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:55.428719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e58cb0-4912-4cff-bdcb-7db47b9ba2e7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:55.429011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e58cb0-4912-4cff-bdcb-7db47b9ba2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:55.429299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e58cb0-4912-4cff-bdcb-7db47b9ba2e7 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:55.430152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e58cb0-4912-4cff-bdcb-7db47b9ba2e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:55.430471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e58cb0-4912-4cff-bdcb-7db47b9ba2e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:55.432009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e58cb0-4912-4cff-bdcb-7db47b9ba2e7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:55.451011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 489/977] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:36:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:55.452179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e58cb0-4912-4cff-bdcb-7db47b9ba2e7 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:55.524410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b38c10-6ac4-4658-ba11-bdc2349e13e8 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:55.547479 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:55.557272 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e58cb0-4912-4cff-bdcb-7db47b9ba2e7 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:55.564420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91a475-a612-4b26-9fad-589c4e563432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:55.565418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91a475-a612-4b26-9fad-589c4e563432 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:55.566034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91a475-a612-4b26-9fad-589c4e563432 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:55.566518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91a475-a612-4b26-9fad-589c4e563432 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:55.567515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91a475-a612-4b26-9fad-589c4e563432 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:55.568134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91a475-a612-4b26-9fad-589c4e563432 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:55.570055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91a475-a612-4b26-9fad-589c4e563432 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:55.594424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9243a-3a83-4941-90fb-e68fbe5babd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:55.595084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9243a-3a83-4941-90fb-e68fbe5babd9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:55.595367 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9243a-3a83-4941-90fb-e68fbe5babd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:55.595652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9243a-3a83-4941-90fb-e68fbe5babd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:55.596547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9243a-3a83-4941-90fb-e68fbe5babd9 None admin] RBAC: Adding query filter params (role.id=79df14b33b3443de8396d1b92c5c1eda, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=063b3fbafe0949d1b09bd9b7b0d70131) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:55.596898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9243a-3a83-4941-90fb-e68fbe5babd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:55.598477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9243a-3a83-4941-90fb-e68fbe5babd9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:55.612631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 490/978] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:36:55 2022] GET /identity/v3/role_assignments?user.id=063b3fbafe0949d1b09bd9b7b0d70131&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=79df14b33b3443de8396d1b92c5c1eda => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:55.621637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 489/979] 10.222.0.47 () {56 vars in 1101 bytes} [Fri Jun 24 23:36:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:56.277983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81262-2dc3-453b-9eea-159e49cd8f73 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:56.278471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81262-2dc3-453b-9eea-159e49cd8f73 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:56.278471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d81262-2dc3-453b-9eea-159e49cd8f73 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:56.280627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 491/980] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:56.285625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138fe58-7732-4e34-85be-ecd4aefdb4d9 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:56.286018 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138fe58-7732-4e34-85be-ecd4aefdb4d9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:56.286188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138fe58-7732-4e34-85be-ecd4aefdb4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:56.303351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-e138fe58-7732-4e34-85be-ecd4aefdb4d9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:56.304889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e138fe58-7732-4e34-85be-ecd4aefdb4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:56.374459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 490/981] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:56.396705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d81262-2dc3-453b-9eea-159e49cd8f73 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:56.447249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:56.502107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d81262-2dc3-453b-9eea-159e49cd8f73 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:56.535638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc7232-8035-4dc7-99d2-7364e735ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:56.536223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc7232-8035-4dc7-99d2-7364e735ce94 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:56.536502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc7232-8035-4dc7-99d2-7364e735ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:56.536801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc7232-8035-4dc7-99d2-7364e735ce94 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:56.537526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc7232-8035-4dc7-99d2-7364e735ce94 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:56.538874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc7232-8035-4dc7-99d2-7364e735ce94 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:56.560618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 492/982] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:36:56 2022] POST /identity/v3/services => generated 242 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:36:56.727268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561ba93-4580-4887-b36b-dd4f83e0f1cb None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:56.727831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561ba93-4580-4887-b36b-dd4f83e0f1cb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:56.728116 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561ba93-4580-4887-b36b-dd4f83e0f1cb None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:56.729724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 491/983] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:56.735271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc7a6d-6686-41ee-9d26-5d88a6912fca None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:56.735567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc7a6d-6686-41ee-9d26-5d88a6912fca None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:56.735879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc7a6d-6686-41ee-9d26-5d88a6912fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:56.753194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-19dc7a6d-6686-41ee-9d26-5d88a6912fca None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:56.754447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-19dc7a6d-6686-41ee-9d26-5d88a6912fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:56.844178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 493/984] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:56.867700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561ba93-4580-4887-b36b-dd4f83e0f1cb None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:56.984653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561ba93-4580-4887-b36b-dd4f83e0f1cb None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:56.994610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:57.021386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99b44e-4165-478f-bf0f-6e690456e56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:57.022188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99b44e-4165-478f-bf0f-6e690456e56f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:57.022469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99b44e-4165-478f-bf0f-6e690456e56f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:57.022995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99b44e-4165-478f-bf0f-6e690456e56f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:57.023649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99b44e-4165-478f-bf0f-6e690456e56f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:57.025054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99b44e-4165-478f-bf0f-6e690456e56f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-2e99b44e-4165-478f-bf0f-6e690456e56f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:57.038596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:57.039578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:36:57.040410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:57.045519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 492/985] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:57.046225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc7a6d-6686-41ee-9d26-5d88a6912fca None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:57.078621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:57.155407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc7a6d-6686-41ee-9d26-5d88a6912fca None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:57.194106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b523ac-a08c-4dc9-86da-664887294c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:57.194868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b523ac-a08c-4dc9-86da-664887294c96 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:57.195383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b523ac-a08c-4dc9-86da-664887294c96 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:57.195897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b523ac-a08c-4dc9-86da-664887294c96 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:57.197050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b523ac-a08c-4dc9-86da-664887294c96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:57.197370 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b523ac-a08c-4dc9-86da-664887294c96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:57.198746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b523ac-a08c-4dc9-86da-664887294c96 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:57.209768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 494/986] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:36:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:57.211064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99b44e-4165-478f-bf0f-6e690456e56f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:57.314458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99b44e-4165-478f-bf0f-6e690456e56f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:57.348747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2d8b-2762-4970-80af-80fb7c41a797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:57.349413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2d8b-2762-4970-80af-80fb7c41a797 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:57.349695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2d8b-2762-4970-80af-80fb7c41a797 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:57.349965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2d8b-2762-4970-80af-80fb7c41a797 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:57.350660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2d8b-2762-4970-80af-80fb7c41a797 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:57.350966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2d8b-2762-4970-80af-80fb7c41a797 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:57.352321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2d8b-2762-4970-80af-80fb7c41a797 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:57.357488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 493/987] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:36:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:57.362988 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b523ac-a08c-4dc9-86da-664887294c96 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:57.455404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:57.463336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b523ac-a08c-4dc9-86da-664887294c96 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:57.502545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec54be-69a0-4a61-a007-ab64b2815939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:57.503273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec54be-69a0-4a61-a007-ab64b2815939 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:57.503552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec54be-69a0-4a61-a007-ab64b2815939 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:57.503823 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec54be-69a0-4a61-a007-ab64b2815939 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:57.504651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec54be-69a0-4a61-a007-ab64b2815939 None admin] RBAC: Adding query filter params (interface=admin, service_id=b5945f129e6e40cfa6cfe32674541fc9, region_id=RegionOne) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:57.504968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec54be-69a0-4a61-a007-ab64b2815939 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:57.506396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec54be-69a0-4a61-a007-ab64b2815939 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:57.516342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 495/988] 10.222.0.47 () {56 vars in 1249 bytes} [Fri Jun 24 23:36:57 2022] GET /identity/v3/endpoints?service_id=b5945f129e6e40cfa6cfe32674541fc9&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:57.522273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2d8b-2762-4970-80af-80fb7c41a797 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:57.558990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:57.625073 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2d8b-2762-4970-80af-80fb7c41a797 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:57.661798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79e054-8fe5-4ffa-b11f-b5a78249e4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:57.662462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79e054-8fe5-4ffa-b11f-b5a78249e4dc None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:57.662741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79e054-8fe5-4ffa-b11f-b5a78249e4dc None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:57.663011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79e054-8fe5-4ffa-b11f-b5a78249e4dc None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:57.663686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79e054-8fe5-4ffa-b11f-b5a78249e4dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:57.665070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79e054-8fe5-4ffa-b11f-b5a78249e4dc None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:57.671658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 494/989] 10.222.0.47 () {56 vars in 1087 bytes} [Fri Jun 24 23:36:57 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:36:57.750967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851722-514e-4a34-b408-9b6b27d11d10 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:57.751164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851722-514e-4a34-b408-9b6b27d11d10 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:57.751432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851722-514e-4a34-b408-9b6b27d11d10 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:57.753125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 496/990] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:57.758543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2ec58-e918-4c8d-9e36-d65bc0a4e2a5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:57.758836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2ec58-e918-4c8d-9e36-d65bc0a4e2a5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:57.759121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2ec58-e918-4c8d-9e36-d65bc0a4e2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:57.777448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f0c2ec58-e918-4c8d-9e36-d65bc0a4e2a5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:57.778690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f0c2ec58-e918-4c8d-9e36-d65bc0a4e2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:57.853201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 495/991] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:57.875484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b851722-514e-4a34-b408-9b6b27d11d10 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:57.984454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b851722-514e-4a34-b408-9b6b27d11d10 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:58.018170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63e963-e9a2-4c29-91ce-67dad849a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:58.018886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63e963-e9a2-4c29-91ce-67dad849a006 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:58.019165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63e963-e9a2-4c29-91ce-67dad849a006 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:58.019439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63e963-e9a2-4c29-91ce-67dad849a006 None admin] PATH_INFO: `/v3/services/image` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:58.020092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63e963-e9a2-4c29-91ce-67dad849a006 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:58.021473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63e963-e9a2-4c29-91ce-67dad849a006 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-be63e963-e9a2-4c29-91ce-67dad849a006 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:58.033785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:58.034795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 23:36:58.035613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:36:58.039806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 497/992] 10.222.0.47 () {56 vars in 1105 bytes} [Fri Jun 24 23:36:57 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:58.040819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2ec58-e918-4c8d-9e36-d65bc0a4e2a5 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:58.070980 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:58.143977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2ec58-e918-4c8d-9e36-d65bc0a4e2a5 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:58.179177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf96503-523c-4d5b-9e67-d713393a9c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:58.179845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf96503-523c-4d5b-9e67-d713393a9c40 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:58.180123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf96503-523c-4d5b-9e67-d713393a9c40 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:58.180393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf96503-523c-4d5b-9e67-d713393a9c40 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:58.181128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf96503-523c-4d5b-9e67-d713393a9c40 None admin] RBAC: Adding query filter params (name=image) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:58.181435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf96503-523c-4d5b-9e67-d713393a9c40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:58.182806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf96503-523c-4d5b-9e67-d713393a9c40 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:58.192436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 496/993] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:58.193990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63e963-e9a2-4c29-91ce-67dad849a006 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:58.219670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:58.295984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63e963-e9a2-4c29-91ce-67dad849a006 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:58.329657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb44a3c-0bc9-4842-a6e4-552f2f645ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:58.330325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb44a3c-0bc9-4842-a6e4-552f2f645ca0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:58.330604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb44a3c-0bc9-4842-a6e4-552f2f645ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:58.330875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb44a3c-0bc9-4842-a6e4-552f2f645ca0 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:58.331583 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb44a3c-0bc9-4842-a6e4-552f2f645ca0 None admin] RBAC: Adding query filter params (type=image) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:58.331890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb44a3c-0bc9-4842-a6e4-552f2f645ca0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:58.333268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb44a3c-0bc9-4842-a6e4-552f2f645ca0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:58.344671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 498/994] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:36:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:58.639376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f13036-edbf-43fe-999c-24665f452bda None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:58.639918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f13036-edbf-43fe-999c-24665f452bda None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:58.639918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f13036-edbf-43fe-999c-24665f452bda None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:58.641460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 497/995] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:58.646404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765ad75-ee50-40ff-9982-2ddc6d772811 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:58.646816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765ad75-ee50-40ff-9982-2ddc6d772811 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:58.646979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765ad75-ee50-40ff-9982-2ddc6d772811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:58.665072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d765ad75-ee50-40ff-9982-2ddc6d772811 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:58.666239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d765ad75-ee50-40ff-9982-2ddc6d772811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:58.731190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 499/996] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:58.753647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f13036-edbf-43fe-999c-24665f452bda None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:58.852366 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:58.861331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f13036-edbf-43fe-999c-24665f452bda None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:58.895469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878652b0-8dca-4438-89eb-9ecdda66085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:58.896190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878652b0-8dca-4438-89eb-9ecdda66085f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:58.896477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878652b0-8dca-4438-89eb-9ecdda66085f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:58.896784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878652b0-8dca-4438-89eb-9ecdda66085f None admin] PATH_INFO: `/v3/services/compute` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:58.897442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878652b0-8dca-4438-89eb-9ecdda66085f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:58.899075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878652b0-8dca-4438-89eb-9ecdda66085f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-878652b0-8dca-4438-89eb-9ecdda66085f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:36:58.911504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:36:58.912531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:36:58.914072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 498/997] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:36:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:36:58.920625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765ad75-ee50-40ff-9982-2ddc6d772811 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:58.940470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:59.030945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765ad75-ee50-40ff-9982-2ddc6d772811 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:59.066082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bd219-4a26-4c86-878a-6bd9096199b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:59.067166 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bd219-4a26-4c86-878a-6bd9096199b0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:59.067649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bd219-4a26-4c86-878a-6bd9096199b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:59.068152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bd219-4a26-4c86-878a-6bd9096199b0 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:59.069165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bd219-4a26-4c86-878a-6bd9096199b0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:59.069674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bd219-4a26-4c86-878a-6bd9096199b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:59.071261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bd219-4a26-4c86-878a-6bd9096199b0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:59.081744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 500/998] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:59.083346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878652b0-8dca-4438-89eb-9ecdda66085f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:59.187314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878652b0-8dca-4438-89eb-9ecdda66085f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:59.222326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabee2d8-7449-47d0-b46d-309ddcdd4311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:59.223369 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabee2d8-7449-47d0-b46d-309ddcdd4311 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:59.223854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabee2d8-7449-47d0-b46d-309ddcdd4311 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:59.224331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabee2d8-7449-47d0-b46d-309ddcdd4311 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:59.225285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabee2d8-7449-47d0-b46d-309ddcdd4311 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:59.225795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabee2d8-7449-47d0-b46d-309ddcdd4311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:59.227426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabee2d8-7449-47d0-b46d-309ddcdd4311 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:59.233611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 499/999] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:36:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:36:59.239613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bd219-4a26-4c86-878a-6bd9096199b0 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:59.323000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:59.344486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bd219-4a26-4c86-878a-6bd9096199b0 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:59.379230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e1351-853d-455e-8274-2bc6e0c37898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:59.379897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e1351-853d-455e-8274-2bc6e0c37898 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:59.380178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e1351-853d-455e-8274-2bc6e0c37898 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:59.380450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e1351-853d-455e-8274-2bc6e0c37898 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:59.381264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e1351-853d-455e-8274-2bc6e0c37898 None admin] RBAC: Adding query filter params (interface=public, service_id=d9ef6875198f4ca68a62a48aebab98ab, region_id=RegionOne) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:36:59.381575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e1351-853d-455e-8274-2bc6e0c37898 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:59.382921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e1351-853d-455e-8274-2bc6e0c37898 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:59.388197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 501/1000] 10.222.0.47 () {56 vars in 1251 bytes} [Fri Jun 24 23:36:59 2022] GET /identity/v3/endpoints?service_id=d9ef6875198f4ca68a62a48aebab98ab&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:36:59.393785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabee2d8-7449-47d0-b46d-309ddcdd4311 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:59.420891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:36:59.499617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabee2d8-7449-47d0-b46d-309ddcdd4311 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:36:59.533269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecafd61-0425-45fb-a5be-2c73ca0b5fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:36:59.533933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecafd61-0425-45fb-a5be-2c73ca0b5fd6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:59.534213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecafd61-0425-45fb-a5be-2c73ca0b5fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:59.534485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecafd61-0425-45fb-a5be-2c73ca0b5fd6 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:59.535157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecafd61-0425-45fb-a5be-2c73ca0b5fd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:36:59.536519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecafd61-0425-45fb-a5be-2c73ca0b5fd6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:36:59.542964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 500/1001] 10.222.0.47 () {56 vars in 1087 bytes} [Fri Jun 24 23:36:59 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:36:59.726110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa8463-cb59-4026-97a4-6a68eb3e30ab None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:59.726616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa8463-cb59-4026-97a4-6a68eb3e30ab None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:59.726707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa8463-cb59-4026-97a4-6a68eb3e30ab None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:59.728012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 502/1002] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:36:59.733939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dcc75-3371-4841-a060-5b5cd098295b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:36:59.734356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dcc75-3371-4841-a060-5b5cd098295b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:36:59.734528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dcc75-3371-4841-a060-5b5cd098295b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:36:59.751807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3f8dcc75-3371-4841-a060-5b5cd098295b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:36:59.752954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3f8dcc75-3371-4841-a060-5b5cd098295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:36:59.827718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 501/1003] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:36:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:36:59.853371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa8463-cb59-4026-97a4-6a68eb3e30ab None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:36:59.966727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa8463-cb59-4026-97a4-6a68eb3e30ab None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:00.001041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a2f3c-a311-4a95-be40-85630438f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:00.001911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a2f3c-a311-4a95-be40-85630438f628 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:00.002145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a2f3c-a311-4a95-be40-85630438f628 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:00.002423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a2f3c-a311-4a95-be40-85630438f628 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:00.003086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a2f3c-a311-4a95-be40-85630438f628 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:00.004452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a2f3c-a311-4a95-be40-85630438f628 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-0a5a2f3c-a311-4a95-be40-85630438f628 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:00.017032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:00.018098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 23:37:00.018927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:00.020371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 503/1004] 10.222.0.47 () {56 vars in 1126 bytes} [Fri Jun 24 23:36:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:00.024914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dcc75-3371-4841-a060-5b5cd098295b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:00.043454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:00.129827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dcc75-3371-4841-a060-5b5cd098295b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:00.164427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c91ee3-a8a2-4135-82ad-d728609d5050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:00.165140 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c91ee3-a8a2-4135-82ad-d728609d5050 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:00.165429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c91ee3-a8a2-4135-82ad-d728609d5050 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:00.165707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c91ee3-a8a2-4135-82ad-d728609d5050 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:00.166430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c91ee3-a8a2-4135-82ad-d728609d5050 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:00.166741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c91ee3-a8a2-4135-82ad-d728609d5050 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:00.168148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c91ee3-a8a2-4135-82ad-d728609d5050 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:00.178892 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 502/1005] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:37:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:00.180512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a2f3c-a311-4a95-be40-85630438f628 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:00.192532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:00.287586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a2f3c-a311-4a95-be40-85630438f628 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:00.322568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381030b7-1709-4a38-ae65-9c82f98ebda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:00.323244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381030b7-1709-4a38-ae65-9c82f98ebda5 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:00.323531 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381030b7-1709-4a38-ae65-9c82f98ebda5 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:00.323811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381030b7-1709-4a38-ae65-9c82f98ebda5 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:00.324524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381030b7-1709-4a38-ae65-9c82f98ebda5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:00.324867 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381030b7-1709-4a38-ae65-9c82f98ebda5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:00.326701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381030b7-1709-4a38-ae65-9c82f98ebda5 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:00.336236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 504/1006] 10.222.0.47 () {56 vars in 1122 bytes} [Fri Jun 24 23:37:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:00.337816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c91ee3-a8a2-4135-82ad-d728609d5050 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:00.407069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405cf30-07fa-473b-b171-bae1e025c009 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:00.407442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405cf30-07fa-473b-b171-bae1e025c009 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:00.407644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405cf30-07fa-473b-b171-bae1e025c009 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:00.409318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 505/1007] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:37:00.414095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5717237-c591-4428-a27d-220ec9e48c63 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:00.414386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5717237-c591-4428-a27d-220ec9e48c63 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:00.414669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5717237-c591-4428-a27d-220ec9e48c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:00.419940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:00.432885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a5717237-c591-4428-a27d-220ec9e48c63 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:00.434083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a5717237-c591-4428-a27d-220ec9e48c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:00.444966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c91ee3-a8a2-4135-82ad-d728609d5050 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:00.480568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef1833-b0d8-4e78-805a-d73d917dd6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:00.481254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1833-b0d8-4e78-805a-d73d917dd6a8 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:00.481542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1833-b0d8-4e78-805a-d73d917dd6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:00.481842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef1833-b0d8-4e78-805a-d73d917dd6a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:00.483056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef1833-b0d8-4e78-805a-d73d917dd6a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:00.484469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef1833-b0d8-4e78-805a-d73d917dd6a8 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:00.506084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:00.508321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 503/1008] 10.222.0.47 () {60 vars in 1144 bytes} [Fri Jun 24 23:37:00 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:37:00.520116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 506/1009] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:37:00.540635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef1833-b0d8-4e78-805a-d73d917dd6a8 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:00.662549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef1833-b0d8-4e78-805a-d73d917dd6a8 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:00.699478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1a8db-885c-4943-8305-ee0866866c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:00.700159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1a8db-885c-4943-8305-ee0866866c72 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:00.700450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1a8db-885c-4943-8305-ee0866866c72 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:00.701395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1a8db-885c-4943-8305-ee0866866c72 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:00.702466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1a8db-885c-4943-8305-ee0866866c72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:00.703853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1a8db-885c-4943-8305-ee0866866c72 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:00.715580 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:00.723139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 504/1010] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:37:00 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:01.639145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f36bf-e921-4052-a7b8-1236850f095a None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:01.639720 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f36bf-e921-4052-a7b8-1236850f095a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:01.639720 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f36bf-e921-4052-a7b8-1236850f095a None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:01.644224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 507/1011] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:01.651779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1afe7-d2ef-4e51-ae12-9d1bee21c47e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:01.652215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1afe7-d2ef-4e51-ae12-9d1bee21c47e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:01.652386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1afe7-d2ef-4e51-ae12-9d1bee21c47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:01.670335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-86e1afe7-d2ef-4e51-ae12-9d1bee21c47e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:01.671537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-86e1afe7-d2ef-4e51-ae12-9d1bee21c47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:01.744478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 505/1012] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:01 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:01.767075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f36bf-e921-4052-a7b8-1236850f095a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:01.869918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f36bf-e921-4052-a7b8-1236850f095a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:01.903723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccebcd-6623-4afb-bc6c-07b28d2ef928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:01.904673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccebcd-6623-4afb-bc6c-07b28d2ef928 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:01.904958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccebcd-6623-4afb-bc6c-07b28d2ef928 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:01.905229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bccebcd-6623-4afb-bc6c-07b28d2ef928 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:01.905877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bccebcd-6623-4afb-bc6c-07b28d2ef928 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:01.907234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bccebcd-6623-4afb-bc6c-07b28d2ef928 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-0bccebcd-6623-4afb-bc6c-07b28d2ef928 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:01.918631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:01.920198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:01.921149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 508/1013] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:37:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:01.928799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1afe7-d2ef-4e51-ae12-9d1bee21c47e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:02.032618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1afe7-d2ef-4e51-ae12-9d1bee21c47e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:02.057467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:02.066833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151211a8-262b-475b-9bcc-b1549baca90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:02.067503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151211a8-262b-475b-9bcc-b1549baca90c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:02.067782 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151211a8-262b-475b-9bcc-b1549baca90c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:02.068056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151211a8-262b-475b-9bcc-b1549baca90c None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:02.068790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151211a8-262b-475b-9bcc-b1549baca90c None admin] RBAC: Adding query filter params (name=compute) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:02.069104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151211a8-262b-475b-9bcc-b1549baca90c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:02.070466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151211a8-262b-475b-9bcc-b1549baca90c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:02.080303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 506/1014] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:37:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:02.081496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccebcd-6623-4afb-bc6c-07b28d2ef928 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:02.143080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:02.182879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bccebcd-6623-4afb-bc6c-07b28d2ef928 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:02.216518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f33f0b-1416-4701-840b-4d40661199f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:02.217393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f33f0b-1416-4701-840b-4d40661199f7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:02.217665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f33f0b-1416-4701-840b-4d40661199f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:02.217936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f33f0b-1416-4701-840b-4d40661199f7 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:02.218642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f33f0b-1416-4701-840b-4d40661199f7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:02.218951 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f33f0b-1416-4701-840b-4d40661199f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:02.220302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f33f0b-1416-4701-840b-4d40661199f7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:02.229673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 509/1015] 10.222.0.47 () {56 vars in 1112 bytes} [Fri Jun 24 23:37:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:02.230872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151211a8-262b-475b-9bcc-b1549baca90c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:02.334654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151211a8-262b-475b-9bcc-b1549baca90c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:02.369015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3822e58-2721-435e-b86a-aad240958583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:02.369682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822e58-2721-435e-b86a-aad240958583 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:02.369971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822e58-2721-435e-b86a-aad240958583 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:02.370252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3822e58-2721-435e-b86a-aad240958583 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:02.370996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3822e58-2721-435e-b86a-aad240958583 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:02.372911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3822e58-2721-435e-b86a-aad240958583 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:02.392831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 507/1016] 10.222.0.47 () {60 vars in 1144 bytes} [Fri Jun 24 23:37:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:02.586872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa223e-6389-426a-84ed-e22eded756fe None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:02.587437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa223e-6389-426a-84ed-e22eded756fe None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:02.587723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa223e-6389-426a-84ed-e22eded756fe None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:02.589114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 510/1017] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:02.595268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac441c-6a93-47e6-856c-862fab580439 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:02.595559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac441c-6a93-47e6-856c-862fab580439 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:02.595839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac441c-6a93-47e6-856c-862fab580439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:02.612524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a3ac441c-6a93-47e6-856c-862fab580439 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:02.613759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a3ac441c-6a93-47e6-856c-862fab580439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:02.659745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:02.694520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 508/1018] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:02.719042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa223e-6389-426a-84ed-e22eded756fe None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:02.768645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890c88c-dbd3-48d6-a535-3ba0897eafcc None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:02.769003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890c88c-dbd3-48d6-a535-3ba0897eafcc None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:02.769219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890c88c-dbd3-48d6-a535-3ba0897eafcc None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:02.770606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 509/1019] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:37:02.775737 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb6331-2a1d-45cb-906d-4e92c62e733f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:02.776021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb6331-2a1d-45cb-906d-4e92c62e733f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:02.776294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb6331-2a1d-45cb-906d-4e92c62e733f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:02.793680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-37fb6331-2a1d-45cb-906d-4e92c62e733f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:02.794884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-37fb6331-2a1d-45cb-906d-4e92c62e733f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:02.831656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa223e-6389-426a-84ed-e22eded756fe None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:02.859523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:02.868349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 510/1020] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:37:02.869290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc34402-8cdc-4f65-bd34-b280da43a36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:02.870082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc34402-8cdc-4f65-bd34-b280da43a36a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:02.870361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc34402-8cdc-4f65-bd34-b280da43a36a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:02.870632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc34402-8cdc-4f65-bd34-b280da43a36a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:02.871786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc34402-8cdc-4f65-bd34-b280da43a36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:02.873371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc34402-8cdc-4f65-bd34-b280da43a36a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:02.879552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 511/1021] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:02.880564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb6331-2a1d-45cb-906d-4e92c62e733f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:02.892293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc34402-8cdc-4f65-bd34-b280da43a36a None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:03.000397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc34402-8cdc-4f65-bd34-b280da43a36a None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:03.015164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb6331-2a1d-45cb-906d-4e92c62e733f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:03.016918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:03.039034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a2331-8058-444e-a785-265166887cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:03.039833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a2331-8058-444e-a785-265166887cfd None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:03.040116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a2331-8058-444e-a785-265166887cfd None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:03.040391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a2331-8058-444e-a785-265166887cfd None admin] PATH_INFO: `/v3/services/image` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:03.041085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a2331-8058-444e-a785-265166887cfd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:03.042444 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a2331-8058-444e-a785-265166887cfd None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:03.054042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e96a97-c811-45ec-b11c-22345461ce20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-9e7a2331-8058-444e-a785-265166887cfd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:03.054624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:03.055579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:03.056667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 512/1022] 10.222.0.47 () {56 vars in 1105 bytes} [Fri Jun 24 23:37:02 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:37:03.058080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e96a97-c811-45ec-b11c-22345461ce20 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:03.058502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e96a97-c811-45ec-b11c-22345461ce20 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:03.058912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e96a97-c811-45ec-b11c-22345461ce20 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:03.059775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e96a97-c811-45ec-b11c-22345461ce20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:03.062673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a2331-8058-444e-a785-265166887cfd None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:03.063769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e96a97-c811-45ec-b11c-22345461ce20 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:03.093152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 511/1023] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:37:02 2022] POST /identity/v3/projects => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:03.135791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:03.183234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a2331-8058-444e-a785-265166887cfd None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:03.220382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211463c-74c9-415f-b7ef-1d7fdf54d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:03.221107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211463c-74c9-415f-b7ef-1d7fdf54d628 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:03.221392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211463c-74c9-415f-b7ef-1d7fdf54d628 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:03.221670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211463c-74c9-415f-b7ef-1d7fdf54d628 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:03.222401 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211463c-74c9-415f-b7ef-1d7fdf54d628 None admin] RBAC: Adding query filter params (name=image) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:03.222712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211463c-74c9-415f-b7ef-1d7fdf54d628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:03.224104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211463c-74c9-415f-b7ef-1d7fdf54d628 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:03.229909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 513/1024] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:03.235414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e96a97-c811-45ec-b11c-22345461ce20 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:03.365762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e96a97-c811-45ec-b11c-22345461ce20 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:03.407517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669776b0-6bd0-415d-8a6d-85bb7930e5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:03.408112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669776b0-6bd0-415d-8a6d-85bb7930e5c4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:03.408398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669776b0-6bd0-415d-8a6d-85bb7930e5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:03.408712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669776b0-6bd0-415d-8a6d-85bb7930e5c4 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:03.409429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669776b0-6bd0-415d-8a6d-85bb7930e5c4 None admin] RBAC: Adding query filter params (type=image) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:03.409739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669776b0-6bd0-415d-8a6d-85bb7930e5c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:03.411103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669776b0-6bd0-415d-8a6d-85bb7930e5c4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:03.420643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 512/1025] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:03.423145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211463c-74c9-415f-b7ef-1d7fdf54d628 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:03.524502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211463c-74c9-415f-b7ef-1d7fdf54d628 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:03.547353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:03.559597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fe283-c72f-49aa-a1c0-cbfac2ebccef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:03.560443 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fe283-c72f-49aa-a1c0-cbfac2ebccef None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:03.560799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fe283-c72f-49aa-a1c0-cbfac2ebccef None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:03.561086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fe283-c72f-49aa-a1c0-cbfac2ebccef None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:03.561882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fe283-c72f-49aa-a1c0-cbfac2ebccef None admin] RBAC: Adding query filter params (interface=public, service_id=33cbed5f05624324abc6c6e52eaf0210, region_id=RegionOne) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:03.562202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fe283-c72f-49aa-a1c0-cbfac2ebccef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:03.563559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fe283-c72f-49aa-a1c0-cbfac2ebccef None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:03.573064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 514/1026] 10.222.0.47 () {56 vars in 1251 bytes} [Fri Jun 24 23:37:03 2022] GET /identity/v3/endpoints?service_id=33cbed5f05624324abc6c6e52eaf0210&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:03.574409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669776b0-6bd0-415d-8a6d-85bb7930e5c4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:03.595640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:03.692079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669776b0-6bd0-415d-8a6d-85bb7930e5c4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:03.729948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c753d4-5689-4873-b0c6-f8afa9286d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:03.730622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c753d4-5689-4873-b0c6-f8afa9286d2b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:03.730908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c753d4-5689-4873-b0c6-f8afa9286d2b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:03.731187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c753d4-5689-4873-b0c6-f8afa9286d2b None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:03.732015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c753d4-5689-4873-b0c6-f8afa9286d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:03.733586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c753d4-5689-4873-b0c6-f8afa9286d2b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:03.740616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 513/1027] 10.222.0.47 () {56 vars in 1087 bytes} [Fri Jun 24 23:37:03 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:37:04.624444 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713d85-eae1-48e1-8329-722c1bffa9f2 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:04.625011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713d85-eae1-48e1-8329-722c1bffa9f2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:04.625090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713d85-eae1-48e1-8329-722c1bffa9f2 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:04.626552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 515/1028] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:04.631748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadb06a-a82a-4e7b-8811-d937ad790478 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:04.632131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadb06a-a82a-4e7b-8811-d937ad790478 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:04.632481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadb06a-a82a-4e7b-8811-d937ad790478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:04.650475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3dadb06a-a82a-4e7b-8811-d937ad790478 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:04.651650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3dadb06a-a82a-4e7b-8811-d937ad790478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:04.719093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 514/1029] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:04.742225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713d85-eae1-48e1-8329-722c1bffa9f2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:04.847325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713d85-eae1-48e1-8329-722c1bffa9f2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:04.885302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9eef9-0dc7-4860-9a7e-a42c5b6557dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:04.886041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9eef9-0dc7-4860-9a7e-a42c5b6557dd None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:04.886324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9eef9-0dc7-4860-9a7e-a42c5b6557dd None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:04.886602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9eef9-0dc7-4860-9a7e-a42c5b6557dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:04.897979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9eef9-0dc7-4860-9a7e-a42c5b6557dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-f6d9eef9-0dc7-4860-9a7e-a42c5b6557dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:04.906138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:04.907054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:04.907883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 516/1030] 10.222.0.47 () {56 vars in 1120 bytes} [Fri Jun 24 23:37:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:04.913371 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadb06a-a82a-4e7b-8811-d937ad790478 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:04.929939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:05.024241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadb06a-a82a-4e7b-8811-d937ad790478 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:05.058149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53208b0b-2e4b-4ba1-9ec5-c8811d2dfba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:05.058812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53208b0b-2e4b-4ba1-9ec5-c8811d2dfba1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.059098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53208b0b-2e4b-4ba1-9ec5-c8811d2dfba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.059377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53208b0b-2e4b-4ba1-9ec5-c8811d2dfba1 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.060154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53208b0b-2e4b-4ba1-9ec5-c8811d2dfba1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:05.060466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53208b0b-2e4b-4ba1-9ec5-c8811d2dfba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:05.061874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53208b0b-2e4b-4ba1-9ec5-c8811d2dfba1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:05.070017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 515/1031] 10.222.0.47 () {56 vars in 1115 bytes} [Fri Jun 24 23:37:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:05.075960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9eef9-0dc7-4860-9a7e-a42c5b6557dd None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:05.124994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:05.179975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9eef9-0dc7-4860-9a7e-a42c5b6557dd None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:05.213941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eeeaea-6e1a-4059-9b78-75693c039ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:05.214732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eeeaea-6e1a-4059-9b78-75693c039ba2 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.215061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eeeaea-6e1a-4059-9b78-75693c039ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.215340 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eeeaea-6e1a-4059-9b78-75693c039ba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.226188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eeeaea-6e1a-4059-9b78-75693c039ba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:05.227616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eeeaea-6e1a-4059-9b78-75693c039ba2 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-61eeeaea-6e1a-4059-9b78-75693c039ba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:37:05.245205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:05.246105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:05.246991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:37:05.246991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:37:05.246991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:05.246991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:05.246991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 517/1032] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:05.253405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53208b0b-2e4b-4ba1-9ec5-c8811d2dfba1 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:05.255502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83237b-6304-42ab-9e08-665c2566eb7a None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.255792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83237b-6304-42ab-9e08-665c2566eb7a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.256230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83237b-6304-42ab-9e08-665c2566eb7a None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.258173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 518/1033] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:37:05.262309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9454e27-8c8f-4dd1-9bf1-9e10e88af420 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.262595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9454e27-8c8f-4dd1-9bf1-9e10e88af420 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.262915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9454e27-8c8f-4dd1-9bf1-9e10e88af420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.280400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-e9454e27-8c8f-4dd1-9bf1-9e10e88af420 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:05.281740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e9454e27-8c8f-4dd1-9bf1-9e10e88af420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:05.336365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:05.357907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 519/1034] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:37:05.373532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53208b0b-2e4b-4ba1-9ec5-c8811d2dfba1 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:05.384429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9454e27-8c8f-4dd1-9bf1-9e10e88af420 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:05.410029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050a44-92b4-48f3-87b8-b582bdcb42a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:05.410707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050a44-92b4-48f3-87b8-b582bdcb42a2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.410998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050a44-92b4-48f3-87b8-b582bdcb42a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.411282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050a44-92b4-48f3-87b8-b582bdcb42a2 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.412012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba050a44-92b4-48f3-87b8-b582bdcb42a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:05.412323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba050a44-92b4-48f3-87b8-b582bdcb42a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:05.413737 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba050a44-92b4-48f3-87b8-b582bdcb42a2 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:05.418215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:05.431178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 516/1035] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:05.432251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050a44-92b4-48f3-87b8-b582bdcb42a2 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:05.504704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9454e27-8c8f-4dd1-9bf1-9e10e88af420 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:05.532912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050a44-92b4-48f3-87b8-b582bdcb42a2 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:05.541129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f2272-3aaa-4850-9938-9f3473d8042d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:05.542274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f2272-3aaa-4850-9938-9f3473d8042d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.542746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f2272-3aaa-4850-9938-9f3473d8042d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.543186 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f2272-3aaa-4850-9938-9f3473d8042d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.545100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f2272-3aaa-4850-9938-9f3473d8042d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:05.546839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f2272-3aaa-4850-9938-9f3473d8042d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:05.549048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 520/1036] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:05.554264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f2272-3aaa-4850-9938-9f3473d8042d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:05.572670 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fda925-98f0-448a-9ef1-d7213b7b8e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:05.573467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fda925-98f0-448a-9ef1-d7213b7b8e4f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.573753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fda925-98f0-448a-9ef1-d7213b7b8e4f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.574029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fda925-98f0-448a-9ef1-d7213b7b8e4f None admin] PATH_INFO: `/v3/users/nova` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.576609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:05.584059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fda925-98f0-448a-9ef1-d7213b7b8e4f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:05.585557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fda925-98f0-448a-9ef1-d7213b7b8e4f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-a8fda925-98f0-448a-9ef1-d7213b7b8e4f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:37:05.601326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:05.602300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:05.603337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 517/1037] 10.222.0.47 () {56 vars in 1093 bytes} [Fri Jun 24 23:37:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:37:05.608561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fda925-98f0-448a-9ef1-d7213b7b8e4f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:05.659540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:05.670440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f2272-3aaa-4850-9938-9f3473d8042d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:05.710311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cd81c-e4dd-404e-b4a0-dc32bc74d389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:05.710900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cd81c-e4dd-404e-b4a0-dc32bc74d389 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.711178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cd81c-e4dd-404e-b4a0-dc32bc74d389 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.711494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cd81c-e4dd-404e-b4a0-dc32bc74d389 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.712212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cd81c-e4dd-404e-b4a0-dc32bc74d389 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:05.713702 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cd81c-e4dd-404e-b4a0-dc32bc74d389 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:05.726175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fda925-98f0-448a-9ef1-d7213b7b8e4f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:05.735944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 521/1038] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:37:05 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:05.765604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f668832-cdb1-43dc-92bc-841cd081e1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:05.766305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f668832-cdb1-43dc-92bc-841cd081e1a5 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.766592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f668832-cdb1-43dc-92bc-841cd081e1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.766871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f668832-cdb1-43dc-92bc-841cd081e1a5 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.767742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f668832-cdb1-43dc-92bc-841cd081e1a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:05.768058 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f668832-cdb1-43dc-92bc-841cd081e1a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:05.769810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f668832-cdb1-43dc-92bc-841cd081e1a5 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:05.796334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 518/1039] 10.222.0.47 () {56 vars in 1133 bytes} [Fri Jun 24 23:37:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:05.797735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cd81c-e4dd-404e-b4a0-dc32bc74d389 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:05.825703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac51b-fa00-4e1c-9c23-bbb8f0ddf04d None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.825703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac51b-fa00-4e1c-9c23-bbb8f0ddf04d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.826115 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ac51b-fa00-4e1c-9c23-bbb8f0ddf04d None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.827511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 519/1040] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:37:05.832948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce13a04-a7a1-4f0f-adad-8aa38751b11b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.833245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce13a04-a7a1-4f0f-adad-8aa38751b11b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.833510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce13a04-a7a1-4f0f-adad-8aa38751b11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.850906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-dce13a04-a7a1-4f0f-adad-8aa38751b11b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:05.852135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-dce13a04-a7a1-4f0f-adad-8aa38751b11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:05.905319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:05.922254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cd81c-e4dd-404e-b4a0-dc32bc74d389 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:05.930264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 520/1041] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:05.953742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce13a04-a7a1-4f0f-adad-8aa38751b11b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:05.960216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565d0c8-499a-4912-b53a-03af1de4d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:05.960979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565d0c8-499a-4912-b53a-03af1de4d1be None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:05.961217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565d0c8-499a-4912-b53a-03af1de4d1be None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:05.962068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565d0c8-499a-4912-b53a-03af1de4d1be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:05.988434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:06.040708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565d0c8-499a-4912-b53a-03af1de4d1be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:06.042179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565d0c8-499a-4912-b53a-03af1de4d1be None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:06.070565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce13a04-a7a1-4f0f-adad-8aa38751b11b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:06.109326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b02d1-b10e-4cca-be1c-383fa6068892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:06.110126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b02d1-b10e-4cca-be1c-383fa6068892 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:06.110414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b02d1-b10e-4cca-be1c-383fa6068892 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:06.110693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b02d1-b10e-4cca-be1c-383fa6068892 None admin] PATH_INFO: `/v3/services/image` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:06.111353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b02d1-b10e-4cca-be1c-383fa6068892 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:06.112763 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b02d1-b10e-4cca-be1c-383fa6068892 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-751b02d1-b10e-4cca-be1c-383fa6068892 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:06.124342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:06.125478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:06.126382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 521/1042] 10.222.0.47 () {56 vars in 1105 bytes} [Fri Jun 24 23:37:05 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:06.131329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b02d1-b10e-4cca-be1c-383fa6068892 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:06.164621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 522/1043] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:37:06.170831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565d0c8-499a-4912-b53a-03af1de4d1be None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:06.183483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:06.224667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:06.239643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b02d1-b10e-4cca-be1c-383fa6068892 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:06.276925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c9e29-132b-4432-af59-82bfcf242d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:06.277598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c9e29-132b-4432-af59-82bfcf242d63 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:06.280022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565d0c8-499a-4912-b53a-03af1de4d1be None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:06.280129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c9e29-132b-4432-af59-82bfcf242d63 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:06.280129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c9e29-132b-4432-af59-82bfcf242d63 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:06.280129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c9e29-132b-4432-af59-82bfcf242d63 None admin] RBAC: Adding query filter params (name=image) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:06.280129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c9e29-132b-4432-af59-82bfcf242d63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:06.281000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c9e29-132b-4432-af59-82bfcf242d63 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:06.290687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 522/1044] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:06.291965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c9e29-132b-4432-af59-82bfcf242d63 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:06.313618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ff26f-eced-4b94-97a7-7dc3a8da96be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:06.314415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ff26f-eced-4b94-97a7-7dc3a8da96be None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:06.314693 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ff26f-eced-4b94-97a7-7dc3a8da96be None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:06.314964 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ff26f-eced-4b94-97a7-7dc3a8da96be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:06.325273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ff26f-eced-4b94-97a7-7dc3a8da96be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:06.327279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ff26f-eced-4b94-97a7-7dc3a8da96be None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-5b4ff26f-eced-4b94-97a7-7dc3a8da96be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:37:06.345532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:06.346481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:06.347377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:37:06.347377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:37:06.347377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:06.347377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:06.351420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 523/1045] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:37:06.352078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ff26f-eced-4b94-97a7-7dc3a8da96be None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:06.403065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c9e29-132b-4432-af59-82bfcf242d63 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:06.414216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:06.439057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc788a47-76cf-4044-a834-8b72518b4e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:06.439729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc788a47-76cf-4044-a834-8b72518b4e6d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:06.440005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc788a47-76cf-4044-a834-8b72518b4e6d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:06.440280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc788a47-76cf-4044-a834-8b72518b4e6d None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:06.441016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc788a47-76cf-4044-a834-8b72518b4e6d None admin] RBAC: Adding query filter params (type=image) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:06.441324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc788a47-76cf-4044-a834-8b72518b4e6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:06.442685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc788a47-76cf-4044-a834-8b72518b4e6d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:06.452623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 523/1046] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:06.453849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc788a47-76cf-4044-a834-8b72518b4e6d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:06.459632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ff26f-eced-4b94-97a7-7dc3a8da96be None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:06.461982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:06.496986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f20ee7-b235-43e5-8826-2ab18c06dee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:06.497774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f20ee7-b235-43e5-8826-2ab18c06dee9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:06.498061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f20ee7-b235-43e5-8826-2ab18c06dee9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:06.498343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f20ee7-b235-43e5-8826-2ab18c06dee9 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:06.499074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f20ee7-b235-43e5-8826-2ab18c06dee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:06.499687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f20ee7-b235-43e5-8826-2ab18c06dee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:06.500793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f20ee7-b235-43e5-8826-2ab18c06dee9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:06.519962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 524/1047] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:06.521450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f20ee7-b235-43e5-8826-2ab18c06dee9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:06.565904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc788a47-76cf-4044-a834-8b72518b4e6d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:06.602573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90e796-6f43-47c2-be22-e762332ebcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:06.603233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90e796-6f43-47c2-be22-e762332ebcd5 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:06.603517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90e796-6f43-47c2-be22-e762332ebcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:06.603910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90e796-6f43-47c2-be22-e762332ebcd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:06.604681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90e796-6f43-47c2-be22-e762332ebcd5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:06.606175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90e796-6f43-47c2-be22-e762332ebcd5 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:06.627948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 524/1048] 10.222.0.47 () {60 vars in 1144 bytes} [Fri Jun 24 23:37:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:06.631126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f20ee7-b235-43e5-8826-2ab18c06dee9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:06.677526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0d47c-a75c-4447-8d8e-28da1fb7cff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:06.678342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0d47c-a75c-4447-8d8e-28da1fb7cff8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:06.678619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0d47c-a75c-4447-8d8e-28da1fb7cff8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:06.678890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0d47c-a75c-4447-8d8e-28da1fb7cff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:06.687481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0d47c-a75c-4447-8d8e-28da1fb7cff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:06.688973 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0d47c-a75c-4447-8d8e-28da1fb7cff8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-f2b0d47c-a75c-4447-8d8e-28da1fb7cff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:37:06.707864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:06.708986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:06.715157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:37:06.715157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:37:06.715157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:37:06.715157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:37:06.715157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:37:06.715157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:37:06.715157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:37:06.715157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:06.715157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 525/1049] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:37:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:37:06.715703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90e796-6f43-47c2-be22-e762332ebcd5 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:06.751478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:06.831707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90e796-6f43-47c2-be22-e762332ebcd5 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:06.866721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233598b1-c9d0-4fec-b3f5-fdb2e4e0fe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:06.867382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233598b1-c9d0-4fec-b3f5-fdb2e4e0fe0c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:06.867663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233598b1-c9d0-4fec-b3f5-fdb2e4e0fe0c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:06.867937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233598b1-c9d0-4fec-b3f5-fdb2e4e0fe0c None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:06.868745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233598b1-c9d0-4fec-b3f5-fdb2e4e0fe0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:06.869056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233598b1-c9d0-4fec-b3f5-fdb2e4e0fe0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:06.870597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233598b1-c9d0-4fec-b3f5-fdb2e4e0fe0c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:06.888186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 525/1050] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:37:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:06.889464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0d47c-a75c-4447-8d8e-28da1fb7cff8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:06.912498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:07.001828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0d47c-a75c-4447-8d8e-28da1fb7cff8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:07.035761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9036212-36fe-4d82-8d69-6d10a1e397dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:07.036414 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9036212-36fe-4d82-8d69-6d10a1e397dc None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:07.036724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9036212-36fe-4d82-8d69-6d10a1e397dc None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:07.036998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9036212-36fe-4d82-8d69-6d10a1e397dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:07.037821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9036212-36fe-4d82-8d69-6d10a1e397dc None admin] RBAC: Adding query filter params (role.id=79df14b33b3443de8396d1b92c5c1eda, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=feb9bec9f1894e179e24a14b81b3f23d) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:07.038127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9036212-36fe-4d82-8d69-6d10a1e397dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:07.039540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9036212-36fe-4d82-8d69-6d10a1e397dc None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:07.045682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 526/1051] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:37:06 2022] GET /identity/v3/role_assignments?user.id=feb9bec9f1894e179e24a14b81b3f23d&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=79df14b33b3443de8396d1b92c5c1eda => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:07.887018 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3acac-a16b-4e95-843e-6e6a29735fa8 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:07.887536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3acac-a16b-4e95-843e-6e6a29735fa8 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:07.887643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3acac-a16b-4e95-843e-6e6a29735fa8 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:07.889639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 526/1052] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:07.895221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732eac8-e32c-4d8e-a1ea-5298306f05f0 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:07.895613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732eac8-e32c-4d8e-a1ea-5298306f05f0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:07.895801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732eac8-e32c-4d8e-a1ea-5298306f05f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:07.912734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-7732eac8-e32c-4d8e-a1ea-5298306f05f0 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:07.913917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7732eac8-e32c-4d8e-a1ea-5298306f05f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:07.987598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 527/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:08.018439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3acac-a16b-4e95-843e-6e6a29735fa8 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:08.115131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:08.128467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3acac-a16b-4e95-843e-6e6a29735fa8 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:08.166923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319c4eb-c10d-47f3-9467-b55cb571c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:08.167638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319c4eb-c10d-47f3-9467-b55cb571c9cf None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:08.167920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319c4eb-c10d-47f3-9467-b55cb571c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:08.168195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319c4eb-c10d-47f3-9467-b55cb571c9cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:08.178886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319c4eb-c10d-47f3-9467-b55cb571c9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-a319c4eb-c10d-47f3-9467-b55cb571c9cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:08.187358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:08.188251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:08.189149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:08.192934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 527/1054] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:37:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:08.194498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732eac8-e32c-4d8e-a1ea-5298306f05f0 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:08.210473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:08.295798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732eac8-e32c-4d8e-a1ea-5298306f05f0 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:08.331170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bedf8-3583-4b3f-9597-025354e446f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:08.331839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bedf8-3583-4b3f-9597-025354e446f3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:08.332123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bedf8-3583-4b3f-9597-025354e446f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:08.332400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bedf8-3583-4b3f-9597-025354e446f3 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:08.333177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bedf8-3583-4b3f-9597-025354e446f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:08.333487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bedf8-3583-4b3f-9597-025354e446f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:08.334840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bedf8-3583-4b3f-9597-025354e446f3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:08.343132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 528/1055] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:37:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:08.348499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319c4eb-c10d-47f3-9467-b55cb571c9cf None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:08.447857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319c4eb-c10d-47f3-9467-b55cb571c9cf None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:08.480990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd81d9-8cc0-4fd3-bcb0-d293a0bee4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:08.482219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd81d9-8cc0-4fd3-bcb0-d293a0bee4cf None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:08.482502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd81d9-8cc0-4fd3-bcb0-d293a0bee4cf None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:08.482780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd81d9-8cc0-4fd3-bcb0-d293a0bee4cf None admin] PATH_INFO: `/v3/users/b1e9cc6bd29e4e14882d8834799368d6` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:08.496190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd81d9-8cc0-4fd3-bcb0-d293a0bee4cf None admin] RBAC: Authorizing `identity:get_user(user_id=b1e9cc6bd29e4e14882d8834799368d6)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:08.498595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd81d9-8cc0-4fd3-bcb0-d293a0bee4cf None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:08.504753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 528/1056] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:08 2022] GET /identity/v3/users/b1e9cc6bd29e4e14882d8834799368d6 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:08.506590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bedf8-3583-4b3f-9597-025354e446f3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:08.586690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:08.612781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bedf8-3583-4b3f-9597-025354e446f3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:08.648523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5317936-3e96-4b24-a469-5a0e573da6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:08.649353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5317936-3e96-4b24-a469-5a0e573da6a6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:08.649733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5317936-3e96-4b24-a469-5a0e573da6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:08.650101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5317936-3e96-4b24-a469-5a0e573da6a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:08.719406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5317936-3e96-4b24-a469-5a0e573da6a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:08.720659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10860db4-e147-47e7-805b-1cc56bc21868 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:08.720957 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10860db4-e147-47e7-805b-1cc56bc21868 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:08.721176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5317936-3e96-4b24-a469-5a0e573da6a6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:08.721289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10860db4-e147-47e7-805b-1cc56bc21868 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:08.722745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 529/1057] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:08.727517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9933d1-5a40-4dc9-b69e-51c6702d3f25 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:08.727809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9933d1-5a40-4dc9-b69e-51c6702d3f25 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:08.728088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9933d1-5a40-4dc9-b69e-51c6702d3f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:08.745289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-af9933d1-5a40-4dc9-b69e-51c6702d3f25 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:08.746497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-af9933d1-5a40-4dc9-b69e-51c6702d3f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:08.749634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:08.808592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:08.825591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 530/1058] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:37:08.843869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 529/1059] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:37:08.845286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9933d1-5a40-4dc9-b69e-51c6702d3f25 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:08.847441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5317936-3e96-4b24-a469-5a0e573da6a6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:08.958054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5317936-3e96-4b24-a469-5a0e573da6a6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:08.964749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9933d1-5a40-4dc9-b69e-51c6702d3f25 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:08.993709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7480544-18c0-4682-9808-f3ab6ebb28b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:08.994595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7480544-18c0-4682-9808-f3ab6ebb28b7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:08.995062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7480544-18c0-4682-9808-f3ab6ebb28b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:08.995422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7480544-18c0-4682-9808-f3ab6ebb28b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.005484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba933d-488a-4fce-86ba-ecbc9190384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:09.006443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba933d-488a-4fce-86ba-ecbc9190384f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.006801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba933d-488a-4fce-86ba-ecbc9190384f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.007150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba933d-488a-4fce-86ba-ecbc9190384f None admin] PATH_INFO: `/v3/projects/591b461f343b4827a4e36ece86489263` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.008395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba933d-488a-4fce-86ba-ecbc9190384f None admin] RBAC: Authorizing `identity:get_project(project_id=591b461f343b4827a4e36ece86489263)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:09.009876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba933d-488a-4fce-86ba-ecbc9190384f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:09.011728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 531/1060] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:08 2022] GET /identity/v3/projects/591b461f343b4827a4e36ece86489263 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:09.017769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba933d-488a-4fce-86ba-ecbc9190384f None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:09.026733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:09.046400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:09.070082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7480544-18c0-4682-9808-f3ab6ebb28b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:09.071517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7480544-18c0-4682-9808-f3ab6ebb28b7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:09.120603 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba933d-488a-4fce-86ba-ecbc9190384f None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:09.155554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba5118-cfec-411c-a345-a697a99a295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:09.156204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba5118-cfec-411c-a345-a697a99a295f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.156489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba5118-cfec-411c-a345-a697a99a295f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.156791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba5118-cfec-411c-a345-a697a99a295f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.157622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba5118-cfec-411c-a345-a697a99a295f None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=591b461f343b4827a4e36ece86489263, user.id=b1e9cc6bd29e4e14882d8834799368d6) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:09.157933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba5118-cfec-411c-a345-a697a99a295f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:09.159340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba5118-cfec-411c-a345-a697a99a295f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:09.177365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 532/1061] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:37:09 2022] GET /identity/v3/role_assignments?user.id=b1e9cc6bd29e4e14882d8834799368d6&scope.project.id=591b461f343b4827a4e36ece86489263&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:09.177365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6548f-cfff-465f-8d64-9690f017f642 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.177365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6548f-cfff-465f-8d64-9690f017f642 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.177365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6548f-cfff-465f-8d64-9690f017f642 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.177365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 533/1062] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:09.177845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 530/1063] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 23:37:09.177927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6548f-cfff-465f-8d64-9690f017f642 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:09.177998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c35df-7f70-4a00-8bbb-9063a8c6f158 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.177998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c35df-7f70-4a00-8bbb-9063a8c6f158 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.177998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c35df-7f70-4a00-8bbb-9063a8c6f158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.194594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-855c35df-7f70-4a00-8bbb-9063a8c6f158 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:09.195823 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-855c35df-7f70-4a00-8bbb-9063a8c6f158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:09.264239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:09.278238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 531/1064] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:09.300427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6548f-cfff-465f-8d64-9690f017f642 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:09.305013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c35df-7f70-4a00-8bbb-9063a8c6f158 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:09.314480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:09.351878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97dd94-0a9f-4e8f-b43e-da5f6bbe398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:09.352986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97dd94-0a9f-4e8f-b43e-da5f6bbe398c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.353271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97dd94-0a9f-4e8f-b43e-da5f6bbe398c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.353788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97dd94-0a9f-4e8f-b43e-da5f6bbe398c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.367234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97dd94-0a9f-4e8f-b43e-da5f6bbe398c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:09.369384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97dd94-0a9f-4e8f-b43e-da5f6bbe398c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-0c97dd94-0a9f-4e8f-b43e-da5f6bbe398c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:37:09.387389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:09.389220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:09.390069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:37:09.390069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:37:09.390069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:09.390069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:09.391205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 534/1065] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:09.396947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97dd94-0a9f-4e8f-b43e-da5f6bbe398c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:09.412244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c35df-7f70-4a00-8bbb-9063a8c6f158 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:09.447088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d87bb-7047-41b4-9869-984e127e97ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:09.447904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d87bb-7047-41b4-9869-984e127e97ca None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.448190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d87bb-7047-41b4-9869-984e127e97ca None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.448474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d87bb-7047-41b4-9869-984e127e97ca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.460127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d87bb-7047-41b4-9869-984e127e97ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-713d87bb-7047-41b4-9869-984e127e97ca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:09.468396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:09.469635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:09.470479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 532/1066] 10.222.0.47 () {56 vars in 1120 bytes} [Fri Jun 24 23:37:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:09.475469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d87bb-7047-41b4-9869-984e127e97ca None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:09.513737 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97dd94-0a9f-4e8f-b43e-da5f6bbe398c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:09.537628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:09.547931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:09.553991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63343755-629b-430f-b6d4-0644526fd0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:09.554665 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63343755-629b-430f-b6d4-0644526fd0cf None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.554956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63343755-629b-430f-b6d4-0644526fd0cf None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.555239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63343755-629b-430f-b6d4-0644526fd0cf None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.555972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63343755-629b-430f-b6d4-0644526fd0cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:09.556287 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63343755-629b-430f-b6d4-0644526fd0cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:09.557700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63343755-629b-430f-b6d4-0644526fd0cf None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:09.570947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 535/1067] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:09.596021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d87bb-7047-41b4-9869-984e127e97ca None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:09.639522 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153e18-284d-4b80-a314-eec7e874f545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:09.640215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153e18-284d-4b80-a314-eec7e874f545 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.640501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153e18-284d-4b80-a314-eec7e874f545 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.640810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153e18-284d-4b80-a314-eec7e874f545 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.641590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153e18-284d-4b80-a314-eec7e874f545 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:09.641904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153e18-284d-4b80-a314-eec7e874f545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:09.643303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153e18-284d-4b80-a314-eec7e874f545 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:09.658460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 533/1068] 10.222.0.47 () {56 vars in 1115 bytes} [Fri Jun 24 23:37:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:09.659499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63343755-629b-430f-b6d4-0644526fd0cf None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:09.771935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63343755-629b-430f-b6d4-0644526fd0cf None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:09.809648 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f2579-28f2-4dc7-9345-cedd06e0aa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:09.810831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f2579-28f2-4dc7-9345-cedd06e0aa2c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.811106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f2579-28f2-4dc7-9345-cedd06e0aa2c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.811373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f2579-28f2-4dc7-9345-cedd06e0aa2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.822220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f2579-28f2-4dc7-9345-cedd06e0aa2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:09.823673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f2579-28f2-4dc7-9345-cedd06e0aa2c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-709f2579-28f2-4dc7-9345-cedd06e0aa2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:37:09.841560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:09.842617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:09.843730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:37:09.843730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:37:09.843730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:09.843730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:09.843730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 536/1069] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:37:09.848803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153e18-284d-4b80-a314-eec7e874f545 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:09.954295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153e18-284d-4b80-a314-eec7e874f545 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:09.981249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:09.989651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22be96-0d12-49ae-9dce-a1fec5b36308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:09.990316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22be96-0d12-49ae-9dce-a1fec5b36308 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:09.990600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22be96-0d12-49ae-9dce-a1fec5b36308 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:09.990877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22be96-0d12-49ae-9dce-a1fec5b36308 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:09.991597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22be96-0d12-49ae-9dce-a1fec5b36308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:09.991907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22be96-0d12-49ae-9dce-a1fec5b36308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:09.993301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22be96-0d12-49ae-9dce-a1fec5b36308 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:10.005242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 534/1070] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:10.010858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f2579-28f2-4dc7-9345-cedd06e0aa2c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:10.045628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:10.109872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f2579-28f2-4dc7-9345-cedd06e0aa2c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:10.142997 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e824e-91fc-4a70-8461-825166c5d51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:10.143783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e824e-91fc-4a70-8461-825166c5d51c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:10.144072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e824e-91fc-4a70-8461-825166c5d51c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:10.144351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e824e-91fc-4a70-8461-825166c5d51c None admin] PATH_INFO: `/v3/users/nova` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:10.151966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e824e-91fc-4a70-8461-825166c5d51c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:10.153451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e824e-91fc-4a70-8461-825166c5d51c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-719e824e-91fc-4a70-8461-825166c5d51c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:37:10.167252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:10.168237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:10.169916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 537/1071] 10.222.0.47 () {56 vars in 1093 bytes} [Fri Jun 24 23:37:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:10.176552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22be96-0d12-49ae-9dce-a1fec5b36308 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:10.274656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22be96-0d12-49ae-9dce-a1fec5b36308 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:10.308986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7e167-bc61-421c-889a-b74c04665e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:10.310356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7e167-bc61-421c-889a-b74c04665e6d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:10.310701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7e167-bc61-421c-889a-b74c04665e6d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:10.310978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7e167-bc61-421c-889a-b74c04665e6d None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:10.311812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7e167-bc61-421c-889a-b74c04665e6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:10.312125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7e167-bc61-421c-889a-b74c04665e6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:10.313560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7e167-bc61-421c-889a-b74c04665e6d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:10.330163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 535/1072] 10.222.0.47 () {56 vars in 1133 bytes} [Fri Jun 24 23:37:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:10.335682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e824e-91fc-4a70-8461-825166c5d51c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:10.429167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:10.436156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e824e-91fc-4a70-8461-825166c5d51c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:10.469074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d7359-2d0a-4516-88e9-f2845cc8b7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:10.469856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d7359-2d0a-4516-88e9-f2845cc8b7e8 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:10.470143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d7359-2d0a-4516-88e9-f2845cc8b7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:10.470424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d7359-2d0a-4516-88e9-f2845cc8b7e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:10.480654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d7359-2d0a-4516-88e9-f2845cc8b7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:10.482377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d7359-2d0a-4516-88e9-f2845cc8b7e8 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-457d7359-2d0a-4516-88e9-f2845cc8b7e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:37:10.499313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:10.500159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:10.501225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:37:10.501225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:37:10.501225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:10.501225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:10.501225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 538/1073] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:10.506146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7e167-bc61-421c-889a-b74c04665e6d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:10.551361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:10.603210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7e167-bc61-421c-889a-b74c04665e6d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:10.637973 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8b832-7ffd-4098-ad46-403e0477744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:10.638653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8b832-7ffd-4098-ad46-403e0477744d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:10.638936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8b832-7ffd-4098-ad46-403e0477744d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:10.639219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8b832-7ffd-4098-ad46-403e0477744d None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:10.639948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8b832-7ffd-4098-ad46-403e0477744d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:10.640256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8b832-7ffd-4098-ad46-403e0477744d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:10.641649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8b832-7ffd-4098-ad46-403e0477744d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:10.653065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 536/1074] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:10.659210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d7359-2d0a-4516-88e9-f2845cc8b7e8 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:10.756928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d7359-2d0a-4516-88e9-f2845cc8b7e8 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:10.790022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417469a9-c554-4b52-86de-fb19aec7fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:10.790816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417469a9-c554-4b52-86de-fb19aec7fd22 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:10.791103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417469a9-c554-4b52-86de-fb19aec7fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:10.791381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417469a9-c554-4b52-86de-fb19aec7fd22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:10.798632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417469a9-c554-4b52-86de-fb19aec7fd22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:10.800077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417469a9-c554-4b52-86de-fb19aec7fd22 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-417469a9-c554-4b52-86de-fb19aec7fd22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:37:10.814122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:10.815168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:10.816250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:37:10.816250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:37:10.816250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:37:10.816250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:37:10.816250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:37:10.816250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:37:10.816250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:37:10.816250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:10.820165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 539/1075] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:10.821047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8b832-7ffd-4098-ad46-403e0477744d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:10.918244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8b832-7ffd-4098-ad46-403e0477744d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:10.925690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:10.952796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c2aa6-cc9b-4ca4-8f73-78f010d265eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:10.953459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c2aa6-cc9b-4ca4-8f73-78f010d265eb None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:10.953743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c2aa6-cc9b-4ca4-8f73-78f010d265eb None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:10.954022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c2aa6-cc9b-4ca4-8f73-78f010d265eb None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:10.954808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c2aa6-cc9b-4ca4-8f73-78f010d265eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:10.955154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c2aa6-cc9b-4ca4-8f73-78f010d265eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:10.956599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c2aa6-cc9b-4ca4-8f73-78f010d265eb None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:10.968901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 537/1076] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:37:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:10.974815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417469a9-c554-4b52-86de-fb19aec7fd22 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:10.984153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:11.074547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417469a9-c554-4b52-86de-fb19aec7fd22 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:11.108462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af7a7b-9b8b-4cae-8423-b63523f01415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:11.109382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af7a7b-9b8b-4cae-8423-b63523f01415 None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:11.109669 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af7a7b-9b8b-4cae-8423-b63523f01415 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:11.109949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af7a7b-9b8b-4cae-8423-b63523f01415 None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/feb9bec9f1894e179e24a14b81b3f23d/roles/79df14b33b3443de8396d1b92c5c1eda` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:11.112145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af7a7b-9b8b-4cae-8423-b63523f01415 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=feb9bec9f1894e179e24a14b81b3f23d, role_id=79df14b33b3443de8396d1b92c5c1eda)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:11.113614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af7a7b-9b8b-4cae-8423-b63523f01415 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:11.127727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 540/1077] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:37:10 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/feb9bec9f1894e179e24a14b81b3f23d/roles/79df14b33b3443de8396d1b92c5c1eda => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:37:11.215011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c299b0-02fc-448d-84f5-e53203cea9b6 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:11.215550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c299b0-02fc-448d-84f5-e53203cea9b6 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:11.215550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c299b0-02fc-448d-84f5-e53203cea9b6 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:11.217397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 538/1078] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:37:11.223183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa932d8-3dcd-4aaa-965b-5619ed514475 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:11.223478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa932d8-3dcd-4aaa-965b-5619ed514475 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:11.223760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa932d8-3dcd-4aaa-965b-5619ed514475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:11.241318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3aa932d8-3dcd-4aaa-965b-5619ed514475 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:11.242550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3aa932d8-3dcd-4aaa-965b-5619ed514475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:11.307380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:11.316794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 541/1079] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:11.339598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c299b0-02fc-448d-84f5-e53203cea9b6 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:11.440279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c299b0-02fc-448d-84f5-e53203cea9b6 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:11.473945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa9c49-4aaa-4717-a5a6-d9e9ad9ff6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:11.474729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa9c49-4aaa-4717-a5a6-d9e9ad9ff6ae None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:11.475007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa9c49-4aaa-4717-a5a6-d9e9ad9ff6ae None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:11.475277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa9c49-4aaa-4717-a5a6-d9e9ad9ff6ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:11.485697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa9c49-4aaa-4717-a5a6-d9e9ad9ff6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-67aa9c49-4aaa-4717-a5a6-d9e9ad9ff6ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:11.493246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:11.494129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:11.494939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 539/1080] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:37:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:11.501103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa932d8-3dcd-4aaa-965b-5619ed514475 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:11.598248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa932d8-3dcd-4aaa-965b-5619ed514475 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:11.631995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36c3e-2ad9-484f-b8df-c94f2761fd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:11.632710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36c3e-2ad9-484f-b8df-c94f2761fd86 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:11.633019 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36c3e-2ad9-484f-b8df-c94f2761fd86 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:11.633301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36c3e-2ad9-484f-b8df-c94f2761fd86 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:11.634055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c36c3e-2ad9-484f-b8df-c94f2761fd86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:11.634365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c36c3e-2ad9-484f-b8df-c94f2761fd86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:11.635745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c36c3e-2ad9-484f-b8df-c94f2761fd86 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:11.644470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 542/1081] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:37:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:11.650421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa9c49-4aaa-4717-a5a6-d9e9ad9ff6ae None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:11.679414 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:11.751080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa9c49-4aaa-4717-a5a6-d9e9ad9ff6ae None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:11.784610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eae12-f02f-49fc-bd2a-ab6eedd1a51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:11.785485 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eae12-f02f-49fc-bd2a-ab6eedd1a51c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:11.785766 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eae12-f02f-49fc-bd2a-ab6eedd1a51c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:11.786039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eae12-f02f-49fc-bd2a-ab6eedd1a51c None admin] PATH_INFO: `/v3/users/b1e9cc6bd29e4e14882d8834799368d6` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:11.787219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eae12-f02f-49fc-bd2a-ab6eedd1a51c None admin] RBAC: Authorizing `identity:get_user(user_id=b1e9cc6bd29e4e14882d8834799368d6)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:11.788668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eae12-f02f-49fc-bd2a-ab6eedd1a51c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:11.790292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 540/1082] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:11 2022] GET /identity/v3/users/b1e9cc6bd29e4e14882d8834799368d6 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:11.795993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36c3e-2ad9-484f-b8df-c94f2761fd86 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:11.853681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:11.895103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36c3e-2ad9-484f-b8df-c94f2761fd86 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:11.927563 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d4600-2d1a-4a2c-934a-f964fc5c363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:11.928417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d4600-2d1a-4a2c-934a-f964fc5c363d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:11.928742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d4600-2d1a-4a2c-934a-f964fc5c363d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:11.929023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d4600-2d1a-4a2c-934a-f964fc5c363d None admin] PATH_INFO: `/v3/projects/591b461f343b4827a4e36ece86489263` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:11.930183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d4600-2d1a-4a2c-934a-f964fc5c363d None admin] RBAC: Authorizing `identity:get_project(project_id=591b461f343b4827a4e36ece86489263)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:11.931643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d4600-2d1a-4a2c-934a-f964fc5c363d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:11.933529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 543/1083] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:11 2022] GET /identity/v3/projects/591b461f343b4827a4e36ece86489263 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:11.939970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eae12-f02f-49fc-bd2a-ab6eedd1a51c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:12.031214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:12.037984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eae12-f02f-49fc-bd2a-ab6eedd1a51c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:12.070541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b7e52-3110-4161-a7d2-fc0b7ff310ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:12.071427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b7e52-3110-4161-a7d2-fc0b7ff310ee None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:12.071708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b7e52-3110-4161-a7d2-fc0b7ff310ee None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:12.071987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b7e52-3110-4161-a7d2-fc0b7ff310ee None admin] PATH_INFO: `/v3/projects/591b461f343b4827a4e36ece86489263/users/b1e9cc6bd29e4e14882d8834799368d6/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:12.073973 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b7e52-3110-4161-a7d2-fc0b7ff310ee None admin] RBAC: Authorizing `identity:create_grant(project_id=591b461f343b4827a4e36ece86489263, user_id=b1e9cc6bd29e4e14882d8834799368d6, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:12.075570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b7e52-3110-4161-a7d2-fc0b7ff310ee None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:12.088028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 541/1084] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:37:11 2022] PUT /identity/v3/projects/591b461f343b4827a4e36ece86489263/users/b1e9cc6bd29e4e14882d8834799368d6/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:37:13.153129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bda3d-5c1a-4ee6-9cc2-10ae24f81f06 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:13.153751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bda3d-5c1a-4ee6-9cc2-10ae24f81f06 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:13.153751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bda3d-5c1a-4ee6-9cc2-10ae24f81f06 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:13.155606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 544/1085] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:13.160905 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294228c-8bc0-48fe-8b22-5f0893995baf None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:13.161312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294228c-8bc0-48fe-8b22-5f0893995baf None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:13.161469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294228c-8bc0-48fe-8b22-5f0893995baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:13.178976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0294228c-8bc0-48fe-8b22-5f0893995baf None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:13.180213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0294228c-8bc0-48fe-8b22-5f0893995baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:13.247018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 542/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:13.270744 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bda3d-5c1a-4ee6-9cc2-10ae24f81f06 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:13.373378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bda3d-5c1a-4ee6-9cc2-10ae24f81f06 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:13.397861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:13.407203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115c0c5-b2e4-4816-80c1-7a1872fbdf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:13.408010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115c0c5-b2e4-4816-80c1-7a1872fbdf04 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:13.408288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115c0c5-b2e4-4816-80c1-7a1872fbdf04 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:13.408560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115c0c5-b2e4-4816-80c1-7a1872fbdf04 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:13.418700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115c0c5-b2e4-4816-80c1-7a1872fbdf04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-6115c0c5-b2e4-4816-80c1-7a1872fbdf04 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:13.426859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:13.427833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:13.428722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 545/1087] 10.222.0.47 () {56 vars in 1120 bytes} [Fri Jun 24 23:37:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:13.434036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294228c-8bc0-48fe-8b22-5f0893995baf None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:13.514410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:13.531694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294228c-8bc0-48fe-8b22-5f0893995baf None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:13.565011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4b523-dab4-43dc-993a-2c6c74013c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:13.565681 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4b523-dab4-43dc-993a-2c6c74013c95 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:13.565967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4b523-dab4-43dc-993a-2c6c74013c95 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:13.566246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4b523-dab4-43dc-993a-2c6c74013c95 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:13.566999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4b523-dab4-43dc-993a-2c6c74013c95 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:13.567309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4b523-dab4-43dc-993a-2c6c74013c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:13.568696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4b523-dab4-43dc-993a-2c6c74013c95 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:13.576640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 543/1088] 10.222.0.47 () {56 vars in 1115 bytes} [Fri Jun 24 23:37:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:13.582244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115c0c5-b2e4-4816-80c1-7a1872fbdf04 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:13.681196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115c0c5-b2e4-4816-80c1-7a1872fbdf04 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:13.713858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b300dcf-660a-48d5-b31d-b35a1eaff454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:13.714667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b300dcf-660a-48d5-b31d-b35a1eaff454 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:13.714937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b300dcf-660a-48d5-b31d-b35a1eaff454 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:13.715245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b300dcf-660a-48d5-b31d-b35a1eaff454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:13.729040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b300dcf-660a-48d5-b31d-b35a1eaff454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:13.730501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b300dcf-660a-48d5-b31d-b35a1eaff454 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-2b300dcf-660a-48d5-b31d-b35a1eaff454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:37:13.748300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:13.749521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:13.755102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:37:13.755102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:37:13.755102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:13.755102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:13.755102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 546/1089] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:37:13.755393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4b523-dab4-43dc-993a-2c6c74013c95 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:13.854666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4b523-dab4-43dc-993a-2c6c74013c95 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:13.886851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3ff41-d0bc-45d9-ad35-c7a4bee42220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:13.888066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3ff41-d0bc-45d9-ad35-c7a4bee42220 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:13.888353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3ff41-d0bc-45d9-ad35-c7a4bee42220 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:13.888662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3ff41-d0bc-45d9-ad35-c7a4bee42220 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:13.889390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3ff41-d0bc-45d9-ad35-c7a4bee42220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:13.889701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3ff41-d0bc-45d9-ad35-c7a4bee42220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:13.891060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3ff41-d0bc-45d9-ad35-c7a4bee42220 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:13.902782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 544/1090] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:13.909252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b300dcf-660a-48d5-b31d-b35a1eaff454 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:13.947931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.012207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b300dcf-660a-48d5-b31d-b35a1eaff454 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:14.046710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa82941-013c-4ba1-ad52-5b3fe89ec0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:14.047507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa82941-013c-4ba1-ad52-5b3fe89ec0f6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.047792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa82941-013c-4ba1-ad52-5b3fe89ec0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.048072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa82941-013c-4ba1-ad52-5b3fe89ec0f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.055464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa82941-013c-4ba1-ad52-5b3fe89ec0f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:14.056942 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa82941-013c-4ba1-ad52-5b3fe89ec0f6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-0fa82941-013c-4ba1-ad52-5b3fe89ec0f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:37:14.071173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:14.072285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:14.073291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 547/1091] 10.222.0.47 () {56 vars in 1093 bytes} [Fri Jun 24 23:37:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:14.079030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3ff41-d0bc-45d9-ad35-c7a4bee42220 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:14.087750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.119607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586a8d5-6de3-420f-919c-229907cff096 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.119793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586a8d5-6de3-420f-919c-229907cff096 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.120070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586a8d5-6de3-420f-919c-229907cff096 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.121637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 548/1092] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:37:14.127036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b844c5-02b1-43d8-9423-b956e39e2fe2 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.127326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b844c5-02b1-43d8-9423-b956e39e2fe2 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.127606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b844c5-02b1-43d8-9423-b956e39e2fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.145377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-56b844c5-02b1-43d8-9423-b956e39e2fe2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:14.146582 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-56b844c5-02b1-43d8-9423-b956e39e2fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:14.186377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3ff41-d0bc-45d9-ad35-c7a4bee42220 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:14.216894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 549/1093] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:37:14.221298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715435f9-44b4-4012-85e6-fc810c7d764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:14.221971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715435f9-44b4-4012-85e6-fc810c7d764d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.222257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715435f9-44b4-4012-85e6-fc810c7d764d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.222538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715435f9-44b4-4012-85e6-fc810c7d764d None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.223379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715435f9-44b4-4012-85e6-fc810c7d764d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:14.223694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715435f9-44b4-4012-85e6-fc810c7d764d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:14.225148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715435f9-44b4-4012-85e6-fc810c7d764d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:14.240780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b844c5-02b1-43d8-9423-b956e39e2fe2 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:14.242527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 545/1094] 10.222.0.47 () {56 vars in 1133 bytes} [Fri Jun 24 23:37:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:14.248449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715435f9-44b4-4012-85e6-fc810c7d764d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:14.268457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.308252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.340764 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b844c5-02b1-43d8-9423-b956e39e2fe2 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:14.346717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715435f9-44b4-4012-85e6-fc810c7d764d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:14.374367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8964-40f5-4d14-ae6d-3207c43a2143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:14.375086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8964-40f5-4d14-ae6d-3207c43a2143 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.375366 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8964-40f5-4d14-ae6d-3207c43a2143 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.375640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a8964-40f5-4d14-ae6d-3207c43a2143 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.381384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3a121-4574-4a79-940b-eab4cbb395b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:14.382033 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3a121-4574-4a79-940b-eab4cbb395b4 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.382319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3a121-4574-4a79-940b-eab4cbb395b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.382601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3a121-4574-4a79-940b-eab4cbb395b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.385746 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a8964-40f5-4d14-ae6d-3207c43a2143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-be3a8964-40f5-4d14-ae6d-3207c43a2143 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:14.393410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:14.394319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:14.395322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 550/1095] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:37:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:37:14.400734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8964-40f5-4d14-ae6d-3207c43a2143 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:14.452495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3a121-4574-4a79-940b-eab4cbb395b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:14.454132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3a121-4574-4a79-940b-eab4cbb395b4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:14.487568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.500514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a8964-40f5-4d14-ae6d-3207c43a2143 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:14.515059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.535123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e60d0-37e5-4f6e-9bc0-824ab5999e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:14.535792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e60d0-37e5-4f6e-9bc0-824ab5999e96 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.536082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e60d0-37e5-4f6e-9bc0-824ab5999e96 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.536360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e60d0-37e5-4f6e-9bc0-824ab5999e96 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.537150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e60d0-37e5-4f6e-9bc0-824ab5999e96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:14.537463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e60d0-37e5-4f6e-9bc0-824ab5999e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:14.538842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e60d0-37e5-4f6e-9bc0-824ab5999e96 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:14.547032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 551/1096] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:37:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:14.552664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e60d0-37e5-4f6e-9bc0-824ab5999e96 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:14.558762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 546/1097] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 23:37:14.559759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3a121-4574-4a79-940b-eab4cbb395b4 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:14.652063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e60d0-37e5-4f6e-9bc0-824ab5999e96 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:14.657171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3a121-4574-4a79-940b-eab4cbb395b4 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:14.685540 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1ea97-40d2-42f0-b68a-3a724990847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:14.686413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1ea97-40d2-42f0-b68a-3a724990847c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.686704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1ea97-40d2-42f0-b68a-3a724990847c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.686979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1ea97-40d2-42f0-b68a-3a724990847c None admin] PATH_INFO: `/v3/users/b1e9cc6bd29e4e14882d8834799368d6` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.688177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1ea97-40d2-42f0-b68a-3a724990847c None admin] RBAC: Authorizing `identity:get_user(user_id=b1e9cc6bd29e4e14882d8834799368d6)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:14.689653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1ea97-40d2-42f0-b68a-3a724990847c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:14.691277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fe162-7a26-4d2c-a255-294f1716772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:14.691425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 552/1098] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:14 2022] GET /identity/v3/users/b1e9cc6bd29e4e14882d8834799368d6 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:14.692063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fe162-7a26-4d2c-a255-294f1716772d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.692342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fe162-7a26-4d2c-a255-294f1716772d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.692644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fe162-7a26-4d2c-a255-294f1716772d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.696881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1ea97-40d2-42f0-b68a-3a724990847c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:14.703431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fe162-7a26-4d2c-a255-294f1716772d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:14.704902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fe162-7a26-4d2c-a255-294f1716772d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:14.716176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-f18fe162-7a26-4d2c-a255-294f1716772d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:37:14.722883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:14.723913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:14.725061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:37:14.725061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:37:14.725061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:37:14.725061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:14.728734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 547/1099] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:14.729443 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fe162-7a26-4d2c-a255-294f1716772d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:14.746769 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.799259 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1ea97-40d2-42f0-b68a-3a724990847c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:14.829327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fe162-7a26-4d2c-a255-294f1716772d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:14.832837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ec76c-04e5-42ff-ae97-0d339541f323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:14.833484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ec76c-04e5-42ff-ae97-0d339541f323 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.833765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ec76c-04e5-42ff-ae97-0d339541f323 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.834035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ec76c-04e5-42ff-ae97-0d339541f323 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.862554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b2929-e0f9-4a21-a9f1-55a16b8d7655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:14.863211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b2929-e0f9-4a21-a9f1-55a16b8d7655 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:14.863489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b2929-e0f9-4a21-a9f1-55a16b8d7655 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:14.863768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b2929-e0f9-4a21-a9f1-55a16b8d7655 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:14.864476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b2929-e0f9-4a21-a9f1-55a16b8d7655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:14.864806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b2929-e0f9-4a21-a9f1-55a16b8d7655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:14.866155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b2929-e0f9-4a21-a9f1-55a16b8d7655 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:14.877756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 548/1100] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:14.883424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b2929-e0f9-4a21-a9f1-55a16b8d7655 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:14.902794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ec76c-04e5-42ff-ae97-0d339541f323 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:14.904742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ec76c-04e5-42ff-ae97-0d339541f323 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:14.921714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.962812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:14.987093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b2929-e0f9-4a21-a9f1-55a16b8d7655 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:15.006621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 553/1101] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 23:37:15.012373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ec76c-04e5-42ff-ae97-0d339541f323 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:15.021051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f762f9-d379-4de1-8a35-14dd40827ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:15.021846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f762f9-d379-4de1-8a35-14dd40827ea9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:15.022122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f762f9-d379-4de1-8a35-14dd40827ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:15.022645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f762f9-d379-4de1-8a35-14dd40827ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:15.030488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f762f9-d379-4de1-8a35-14dd40827ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:15.031920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f762f9-d379-4de1-8a35-14dd40827ea9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-29f762f9-d379-4de1-8a35-14dd40827ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:37:15.046456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:15.047498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:15.048411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:37:15.048411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:37:15.048411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:37:15.048411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:37:15.048411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:37:15.048411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:37:15.048411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:37:15.048411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:15.048411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 549/1102] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:37:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:15.053202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f762f9-d379-4de1-8a35-14dd40827ea9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:15.110593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ec76c-04e5-42ff-ae97-0d339541f323 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:15.128280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:15.145298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395d565-f366-46f8-8fa2-f82a599dc686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:15.146157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395d565-f366-46f8-8fa2-f82a599dc686 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:15.146443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395d565-f366-46f8-8fa2-f82a599dc686 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:15.146715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395d565-f366-46f8-8fa2-f82a599dc686 None admin] PATH_INFO: `/v3/projects/591b461f343b4827a4e36ece86489263` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:15.148129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395d565-f366-46f8-8fa2-f82a599dc686 None admin] RBAC: Authorizing `identity:get_project(project_id=591b461f343b4827a4e36ece86489263)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:15.149591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395d565-f366-46f8-8fa2-f82a599dc686 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:15.151277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 554/1103] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:15 2022] GET /identity/v3/projects/591b461f343b4827a4e36ece86489263 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:15.151381 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f762f9-d379-4de1-8a35-14dd40827ea9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:15.157942 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395d565-f366-46f8-8fa2-f82a599dc686 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:15.190131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71305f39-2060-497f-9eb8-c10074c4b457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:15.190786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71305f39-2060-497f-9eb8-c10074c4b457 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:15.191065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71305f39-2060-497f-9eb8-c10074c4b457 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:15.191342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71305f39-2060-497f-9eb8-c10074c4b457 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:15.192126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71305f39-2060-497f-9eb8-c10074c4b457 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:15.192434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71305f39-2060-497f-9eb8-c10074c4b457 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:15.193873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71305f39-2060-497f-9eb8-c10074c4b457 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:15.196316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:15.206432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 550/1104] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:37:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:15.212216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71305f39-2060-497f-9eb8-c10074c4b457 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:15.255340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395d565-f366-46f8-8fa2-f82a599dc686 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:15.288897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d84213-2302-4381-a96a-f1bd1e02ba64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:15.289550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d84213-2302-4381-a96a-f1bd1e02ba64 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:15.289832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d84213-2302-4381-a96a-f1bd1e02ba64 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:15.290117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d84213-2302-4381-a96a-f1bd1e02ba64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:15.290945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d84213-2302-4381-a96a-f1bd1e02ba64 None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=591b461f343b4827a4e36ece86489263, user.id=b1e9cc6bd29e4e14882d8834799368d6) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:15.291256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d84213-2302-4381-a96a-f1bd1e02ba64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:15.292708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d84213-2302-4381-a96a-f1bd1e02ba64 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:15.299027 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 555/1105] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:37:15 2022] GET /identity/v3/role_assignments?user.id=b1e9cc6bd29e4e14882d8834799368d6&scope.project.id=591b461f343b4827a4e36ece86489263&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:15.313626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71305f39-2060-497f-9eb8-c10074c4b457 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:15.352034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d0a76-aa4e-4e2d-93a2-e63fe84ee2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:15.352768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d0a76-aa4e-4e2d-93a2-e63fe84ee2b0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:15.353060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d0a76-aa4e-4e2d-93a2-e63fe84ee2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:15.353342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d0a76-aa4e-4e2d-93a2-e63fe84ee2b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:15.354211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d0a76-aa4e-4e2d-93a2-e63fe84ee2b0 None admin] RBAC: Adding query filter params (role.id=79df14b33b3443de8396d1b92c5c1eda, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=feb9bec9f1894e179e24a14b81b3f23d) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:15.354529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d0a76-aa4e-4e2d-93a2-e63fe84ee2b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:15.356026 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d0a76-aa4e-4e2d-93a2-e63fe84ee2b0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:15.376627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 551/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:37:15 2022] GET /identity/v3/role_assignments?user.id=feb9bec9f1894e179e24a14b81b3f23d&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=79df14b33b3443de8396d1b92c5c1eda => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:17.403977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3a17c-234f-4009-bebf-f624cd65c04f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:17.404561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3a17c-234f-4009-bebf-f624cd65c04f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:17.404561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3a17c-234f-4009-bebf-f624cd65c04f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:17.405939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 556/1107] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:17.411121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291398ef-a8de-4485-a8f0-352857198b10 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:17.411536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291398ef-a8de-4485-a8f0-352857198b10 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:17.411678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291398ef-a8de-4485-a8f0-352857198b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:17.428747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-291398ef-a8de-4485-a8f0-352857198b10 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:17.429942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-291398ef-a8de-4485-a8f0-352857198b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:17.483502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:17.494802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 552/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:17.520151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3a17c-234f-4009-bebf-f624cd65c04f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:17.571237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:17.620973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df3a17c-234f-4009-bebf-f624cd65c04f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:17.655734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81276323-7915-40aa-93b8-35a4fd39cf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:17.656451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81276323-7915-40aa-93b8-35a4fd39cf35 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:17.656765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81276323-7915-40aa-93b8-35a4fd39cf35 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:17.657043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81276323-7915-40aa-93b8-35a4fd39cf35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:17.658200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81276323-7915-40aa-93b8-35a4fd39cf35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:17.659659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81276323-7915-40aa-93b8-35a4fd39cf35 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:17.661296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 557/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:17.666874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291398ef-a8de-4485-a8f0-352857198b10 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:17.768747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291398ef-a8de-4485-a8f0-352857198b10 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:17.802023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711eed1-75c2-4e1f-bee4-aeb4d7e67b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:17.802685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711eed1-75c2-4e1f-bee4-aeb4d7e67b06 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:17.802969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711eed1-75c2-4e1f-bee4-aeb4d7e67b06 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:17.803247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711eed1-75c2-4e1f-bee4-aeb4d7e67b06 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:17.803959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711eed1-75c2-4e1f-bee4-aeb4d7e67b06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:17.805411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711eed1-75c2-4e1f-bee4-aeb4d7e67b06 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:17.825743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 553/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:37:17 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:37:19.910616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50dea0-f891-45f2-b1a4-aac8a7a9f16c None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:19.911190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50dea0-f891-45f2-b1a4-aac8a7a9f16c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:19.911190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f50dea0-f891-45f2-b1a4-aac8a7a9f16c None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:19.912538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 558/1111] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:19.917675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54b56f-2553-440e-994a-a28c0a4b8505 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:19.918080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54b56f-2553-440e-994a-a28c0a4b8505 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:19.918237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54b56f-2553-440e-994a-a28c0a4b8505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:19.936071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-5f54b56f-2553-440e-994a-a28c0a4b8505 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:19.937302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5f54b56f-2553-440e-994a-a28c0a4b8505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:19.978357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:20.004036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 554/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:20.028004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50dea0-f891-45f2-b1a4-aac8a7a9f16c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:20.126093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f50dea0-f891-45f2-b1a4-aac8a7a9f16c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:20.142579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:20.159943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98851da7-be2c-4ce9-8cd5-1350f80da782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:20.160882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98851da7-be2c-4ce9-8cd5-1350f80da782 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:20.161119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98851da7-be2c-4ce9-8cd5-1350f80da782 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:20.161396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98851da7-be2c-4ce9-8cd5-1350f80da782 None admin] PATH_INFO: `/v3/roles/32433ed911104bb68970c0a00d0ddd49` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:20.162853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98851da7-be2c-4ce9-8cd5-1350f80da782 None admin] RBAC: Authorizing `identity:get_role(role_id=32433ed911104bb68970c0a00d0ddd49)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:20.164237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98851da7-be2c-4ce9-8cd5-1350f80da782 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:20.165636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 559/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:20 2022] GET /identity/v3/roles/32433ed911104bb68970c0a00d0ddd49 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:20.171947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54b56f-2553-440e-994a-a28c0a4b8505 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:20.267535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54b56f-2553-440e-994a-a28c0a4b8505 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:20.300455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6dd09-584c-493b-8461-e0bf96110bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:20.301345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6dd09-584c-493b-8461-e0bf96110bfb None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:20.301634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6dd09-584c-493b-8461-e0bf96110bfb None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:20.301913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6dd09-584c-493b-8461-e0bf96110bfb None admin] PATH_INFO: `/v3/users/e98bc43f7a7044ba875564e825ab13b2` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:20.314948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6dd09-584c-493b-8461-e0bf96110bfb None admin] RBAC: Authorizing `identity:get_user(user_id=e98bc43f7a7044ba875564e825ab13b2)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:20.316380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6dd09-584c-493b-8461-e0bf96110bfb None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:20.318034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 555/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:20 2022] GET /identity/v3/users/e98bc43f7a7044ba875564e825ab13b2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:20.323554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98851da7-be2c-4ce9-8cd5-1350f80da782 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:20.420529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98851da7-be2c-4ce9-8cd5-1350f80da782 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:20.453842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3a337-fad5-4d55-953d-b5dc1f891391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:20.454482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3a337-fad5-4d55-953d-b5dc1f891391 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:20.454770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3a337-fad5-4d55-953d-b5dc1f891391 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:20.455053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3a337-fad5-4d55-953d-b5dc1f891391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:20.507326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:20.520466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3a337-fad5-4d55-953d-b5dc1f891391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:20.521940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3a337-fad5-4d55-953d-b5dc1f891391 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:20.616855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 560/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:37:20.622638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6dd09-584c-493b-8461-e0bf96110bfb None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:20.658535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:20.723810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6dd09-584c-493b-8461-e0bf96110bfb None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:20.759639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6bbf3-bf14-4d91-a7f9-8d1a408fec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:20.760497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bbf3-bf14-4d91-a7f9-8d1a408fec62 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:20.760807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bbf3-bf14-4d91-a7f9-8d1a408fec62 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:20.761086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6bbf3-bf14-4d91-a7f9-8d1a408fec62 None admin] PATH_INFO: `/v3/projects/591b461f343b4827a4e36ece86489263` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:20.762265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6bbf3-bf14-4d91-a7f9-8d1a408fec62 None admin] RBAC: Authorizing `identity:get_project(project_id=591b461f343b4827a4e36ece86489263)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:20.763791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6bbf3-bf14-4d91-a7f9-8d1a408fec62 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:20.765518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 556/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:20 2022] GET /identity/v3/projects/591b461f343b4827a4e36ece86489263 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:20.771598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3a337-fad5-4d55-953d-b5dc1f891391 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:20.852063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:20.867107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3a337-fad5-4d55-953d-b5dc1f891391 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:20.899915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6d4d5-026d-49ec-aa69-7e39df842222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:20.900551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6d4d5-026d-49ec-aa69-7e39df842222 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:20.900859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6d4d5-026d-49ec-aa69-7e39df842222 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:20.901131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6d4d5-026d-49ec-aa69-7e39df842222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:20.901943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6d4d5-026d-49ec-aa69-7e39df842222 None admin] RBAC: Adding query filter params (role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=591b461f343b4827a4e36ece86489263, user.id=e98bc43f7a7044ba875564e825ab13b2) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:20.902248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6d4d5-026d-49ec-aa69-7e39df842222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:20.903653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6d4d5-026d-49ec-aa69-7e39df842222 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:20.909551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 561/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:37:20 2022] GET /identity/v3/role_assignments?user.id=e98bc43f7a7044ba875564e825ab13b2&scope.project.id=591b461f343b4827a4e36ece86489263&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:22.901398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36dca3-89d8-494a-b5a7-db93da73a7c2 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:22.901928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36dca3-89d8-494a-b5a7-db93da73a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:22.901928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36dca3-89d8-494a-b5a7-db93da73a7c2 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:22.903305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 557/1118] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:22.908529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811a62e-5901-4c6b-93d9-323b57bc0277 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:22.908979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811a62e-5901-4c6b-93d9-323b57bc0277 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:22.909118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811a62e-5901-4c6b-93d9-323b57bc0277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:22.925963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7811a62e-5901-4c6b-93d9-323b57bc0277 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:22.927151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7811a62e-5901-4c6b-93d9-323b57bc0277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:22.992682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 562/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:23.016777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36dca3-89d8-494a-b5a7-db93da73a7c2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:23.137296 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36dca3-89d8-494a-b5a7-db93da73a7c2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:23.138798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:23.176240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f342a-bfad-42f5-9979-4bb1ab964aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:23.177080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f342a-bfad-42f5-9979-4bb1ab964aff None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:23.177388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f342a-bfad-42f5-9979-4bb1ab964aff None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:23.177664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f342a-bfad-42f5-9979-4bb1ab964aff None admin] PATH_INFO: `/v3/roles/32433ed911104bb68970c0a00d0ddd49` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:23.179263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f342a-bfad-42f5-9979-4bb1ab964aff None admin] RBAC: Authorizing `identity:get_role(role_id=32433ed911104bb68970c0a00d0ddd49)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:23.180648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f342a-bfad-42f5-9979-4bb1ab964aff None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:23.182022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 558/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:23 2022] GET /identity/v3/roles/32433ed911104bb68970c0a00d0ddd49 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:23.189871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811a62e-5901-4c6b-93d9-323b57bc0277 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:23.266644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:23.287053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811a62e-5901-4c6b-93d9-323b57bc0277 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:23.321628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f556a-270a-4c84-8452-de15245b3e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:23.322500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f556a-270a-4c84-8452-de15245b3e15 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:23.322778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f556a-270a-4c84-8452-de15245b3e15 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:23.323048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f556a-270a-4c84-8452-de15245b3e15 None admin] PATH_INFO: `/v3/users/e98bc43f7a7044ba875564e825ab13b2` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:23.324220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f556a-270a-4c84-8452-de15245b3e15 None admin] RBAC: Authorizing `identity:get_user(user_id=e98bc43f7a7044ba875564e825ab13b2)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:23.325671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f556a-270a-4c84-8452-de15245b3e15 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:23.327461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 563/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:23 2022] GET /identity/v3/users/e98bc43f7a7044ba875564e825ab13b2 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:23.333708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f342a-bfad-42f5-9979-4bb1ab964aff None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:23.453834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f342a-bfad-42f5-9979-4bb1ab964aff None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:23.496726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac93a40-ed62-49e3-9d6b-cf266d680f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:23.497582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac93a40-ed62-49e3-9d6b-cf266d680f16 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:23.497976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac93a40-ed62-49e3-9d6b-cf266d680f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:23.498429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac93a40-ed62-49e3-9d6b-cf266d680f16 None admin] PATH_INFO: `/v3/projects/591b461f343b4827a4e36ece86489263` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:23.499703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac93a40-ed62-49e3-9d6b-cf266d680f16 None admin] RBAC: Authorizing `identity:get_project(project_id=591b461f343b4827a4e36ece86489263)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:23.501181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac93a40-ed62-49e3-9d6b-cf266d680f16 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:23.502889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 559/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:23 2022] GET /identity/v3/projects/591b461f343b4827a4e36ece86489263 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:23.508890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f556a-270a-4c84-8452-de15245b3e15 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:23.609279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f556a-270a-4c84-8452-de15245b3e15 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:23.642878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bd1a8-c9ec-470c-bca2-09894e4f81f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:23.643682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bd1a8-c9ec-470c-bca2-09894e4f81f9 None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:23.643962 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bd1a8-c9ec-470c-bca2-09894e4f81f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:23.644237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bd1a8-c9ec-470c-bca2-09894e4f81f9 None admin] PATH_INFO: `/v3/projects/591b461f343b4827a4e36ece86489263/users/e98bc43f7a7044ba875564e825ab13b2/roles/32433ed911104bb68970c0a00d0ddd49` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:23.646444 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bd1a8-c9ec-470c-bca2-09894e4f81f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=591b461f343b4827a4e36ece86489263, user_id=e98bc43f7a7044ba875564e825ab13b2, role_id=32433ed911104bb68970c0a00d0ddd49)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:23.648127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bd1a8-c9ec-470c-bca2-09894e4f81f9 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:23.662905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 564/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:37:23 2022] PUT /identity/v3/projects/591b461f343b4827a4e36ece86489263/users/e98bc43f7a7044ba875564e825ab13b2/roles/32433ed911104bb68970c0a00d0ddd49 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:37:25.724045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596ec3c-63f1-446f-9465-82444d430f52 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:25.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596ec3c-63f1-446f-9465-82444d430f52 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:25.724691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596ec3c-63f1-446f-9465-82444d430f52 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:25.725975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 560/1124] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:25.731025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32544df9-a6c3-4da6-a912-b76380cc5981 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:25.731462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32544df9-a6c3-4da6-a912-b76380cc5981 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:25.731556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32544df9-a6c3-4da6-a912-b76380cc5981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:25.748676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-32544df9-a6c3-4da6-a912-b76380cc5981 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:25.749851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-32544df9-a6c3-4da6-a912-b76380cc5981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:25.751762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:25.818383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 565/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:25.842882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596ec3c-63f1-446f-9465-82444d430f52 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:25.882350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:25.942614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596ec3c-63f1-446f-9465-82444d430f52 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:25.977390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07014e9-3a9c-422f-9e9e-320d3dd0f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:25.978185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07014e9-3a9c-422f-9e9e-320d3dd0f37e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:25.978470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07014e9-3a9c-422f-9e9e-320d3dd0f37e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:25.978748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07014e9-3a9c-422f-9e9e-320d3dd0f37e None admin] PATH_INFO: `/v3/roles/32433ed911104bb68970c0a00d0ddd49` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:25.980219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07014e9-3a9c-422f-9e9e-320d3dd0f37e None admin] RBAC: Authorizing `identity:get_role(role_id=32433ed911104bb68970c0a00d0ddd49)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:25.981609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07014e9-3a9c-422f-9e9e-320d3dd0f37e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:25.983878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 561/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:25 2022] GET /identity/v3/roles/32433ed911104bb68970c0a00d0ddd49 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:25.989598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32544df9-a6c3-4da6-a912-b76380cc5981 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:26.085938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32544df9-a6c3-4da6-a912-b76380cc5981 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:26.119396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ffdc1-df2a-4d7d-b9c8-97ed5e708f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:26.120251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ffdc1-df2a-4d7d-b9c8-97ed5e708f10 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:26.120533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ffdc1-df2a-4d7d-b9c8-97ed5e708f10 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:26.120837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ffdc1-df2a-4d7d-b9c8-97ed5e708f10 None admin] PATH_INFO: `/v3/users/e98bc43f7a7044ba875564e825ab13b2` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:26.122008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ffdc1-df2a-4d7d-b9c8-97ed5e708f10 None admin] RBAC: Authorizing `identity:get_user(user_id=e98bc43f7a7044ba875564e825ab13b2)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:26.123980 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ffdc1-df2a-4d7d-b9c8-97ed5e708f10 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:26.125612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 566/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:25 2022] GET /identity/v3/users/e98bc43f7a7044ba875564e825ab13b2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:26.131071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07014e9-3a9c-422f-9e9e-320d3dd0f37e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:26.227927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07014e9-3a9c-422f-9e9e-320d3dd0f37e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:26.234912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:26.260928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c12a52-c141-4eb8-ae45-a69bb581327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:26.261566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c12a52-c141-4eb8-ae45-a69bb581327d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:26.261849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c12a52-c141-4eb8-ae45-a69bb581327d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:26.262124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c12a52-c141-4eb8-ae45-a69bb581327d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:26.326831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c12a52-c141-4eb8-ae45-a69bb581327d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:26.328255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c12a52-c141-4eb8-ae45-a69bb581327d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:26.421849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 562/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:37:26.427679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ffdc1-df2a-4d7d-b9c8-97ed5e708f10 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:26.437001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:26.525559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ffdc1-df2a-4d7d-b9c8-97ed5e708f10 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:26.560592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42e184-bdd2-4ecc-b901-1b2e2b15064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:26.561454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42e184-bdd2-4ecc-b901-1b2e2b15064f None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:26.561755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42e184-bdd2-4ecc-b901-1b2e2b15064f None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:26.562033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42e184-bdd2-4ecc-b901-1b2e2b15064f None admin] PATH_INFO: `/v3/projects/591b461f343b4827a4e36ece86489263` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:26.563215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42e184-bdd2-4ecc-b901-1b2e2b15064f None admin] RBAC: Authorizing `identity:get_project(project_id=591b461f343b4827a4e36ece86489263)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:26.564660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42e184-bdd2-4ecc-b901-1b2e2b15064f None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:26.566341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 567/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:26 2022] GET /identity/v3/projects/591b461f343b4827a4e36ece86489263 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:26.572349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c12a52-c141-4eb8-ae45-a69bb581327d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:26.580750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:26.668499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c12a52-c141-4eb8-ae45-a69bb581327d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:26.701163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358c301-d4cc-4ebd-ba92-c26330514a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:26.701949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358c301-d4cc-4ebd-ba92-c26330514a44 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:26.702233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358c301-d4cc-4ebd-ba92-c26330514a44 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:26.702513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358c301-d4cc-4ebd-ba92-c26330514a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:26.703335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358c301-d4cc-4ebd-ba92-c26330514a44 None admin] RBAC: Adding query filter params (role.id=32433ed911104bb68970c0a00d0ddd49, scope.project.id=591b461f343b4827a4e36ece86489263, user.id=e98bc43f7a7044ba875564e825ab13b2) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:26.703649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358c301-d4cc-4ebd-ba92-c26330514a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:26.705091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358c301-d4cc-4ebd-ba92-c26330514a44 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:26.710710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 563/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:37:26 2022] GET /identity/v3/role_assignments?user.id=e98bc43f7a7044ba875564e825ab13b2&scope.project.id=591b461f343b4827a4e36ece86489263&role.id=32433ed911104bb68970c0a00d0ddd49 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:28.766813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c92a31-bc98-40f1-9664-fb4bd842b93c None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:28.767367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c92a31-bc98-40f1-9664-fb4bd842b93c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:28.767367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c92a31-bc98-40f1-9664-fb4bd842b93c None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:28.768820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 568/1131] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:28.774129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035cb4f-7b99-4ebd-a49e-a4b6bfa4827c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:28.774518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035cb4f-7b99-4ebd-a49e-a4b6bfa4827c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:28.774684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035cb4f-7b99-4ebd-a49e-a4b6bfa4827c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:28.791647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f035cb4f-7b99-4ebd-a49e-a4b6bfa4827c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:28.792868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f035cb4f-7b99-4ebd-a49e-a4b6bfa4827c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:28.853263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:28.858145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 564/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:37:28.881171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c92a31-bc98-40f1-9664-fb4bd842b93c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:28.955195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:28.980683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c92a31-bc98-40f1-9664-fb4bd842b93c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:29.013970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faad52-a6d7-4c15-86b3-58b05b48c820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:29.014743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faad52-a6d7-4c15-86b3-58b05b48c820 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:29.015022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faad52-a6d7-4c15-86b3-58b05b48c820 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:29.015293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faad52-a6d7-4c15-86b3-58b05b48c820 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:29.016445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2faad52-a6d7-4c15-86b3-58b05b48c820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:29.017948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2faad52-a6d7-4c15-86b3-58b05b48c820 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:29.019488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 569/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:29.025576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035cb4f-7b99-4ebd-a49e-a4b6bfa4827c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:29.121573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035cb4f-7b99-4ebd-a49e-a4b6bfa4827c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:29.154902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08445f2a-f532-49fa-ad87-08351fdfc6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:29.155550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08445f2a-f532-49fa-ad87-08351fdfc6e8 None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:29.155830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08445f2a-f532-49fa-ad87-08351fdfc6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:29.156105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08445f2a-f532-49fa-ad87-08351fdfc6e8 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:29.156830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08445f2a-f532-49fa-ad87-08351fdfc6e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:29.158246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08445f2a-f532-49fa-ad87-08351fdfc6e8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:29.183394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 565/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:37:29 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:31.302576 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeb201-c0a5-4b73-8b33-b53879fd1179 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:31.303091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeb201-c0a5-4b73-8b33-b53879fd1179 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:31.303091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeb201-c0a5-4b73-8b33-b53879fd1179 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:31.304398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 570/1135] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:31.309754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f90674-8be9-4452-8a97-248177cce5a2 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:31.310169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f90674-8be9-4452-8a97-248177cce5a2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:31.310407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f90674-8be9-4452-8a97-248177cce5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:31.327518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-74f90674-8be9-4452-8a97-248177cce5a2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:31.328733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-74f90674-8be9-4452-8a97-248177cce5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:31.377215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:31.398685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 566/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:31.426571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeb201-c0a5-4b73-8b33-b53879fd1179 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:31.525335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeb201-c0a5-4b73-8b33-b53879fd1179 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:31.558614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78afa88-e00e-4f17-b4f1-3c1d1b7704f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:31.559337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78afa88-e00e-4f17-b4f1-3c1d1b7704f8 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:31.559624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78afa88-e00e-4f17-b4f1-3c1d1b7704f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:31.559947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78afa88-e00e-4f17-b4f1-3c1d1b7704f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:31.561315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78afa88-e00e-4f17-b4f1-3c1d1b7704f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:31.562742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78afa88-e00e-4f17-b4f1-3c1d1b7704f8 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:31.564353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 571/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:37:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:31.570314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f90674-8be9-4452-8a97-248177cce5a2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:31.664951 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f90674-8be9-4452-8a97-248177cce5a2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:31.698919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee237c-d444-444d-a0b5-9c841236f46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:31.699591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee237c-d444-444d-a0b5-9c841236f46f None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:31.699873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee237c-d444-444d-a0b5-9c841236f46f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:31.700152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee237c-d444-444d-a0b5-9c841236f46f None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:31.700899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee237c-d444-444d-a0b5-9c841236f46f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:31.702335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee237c-d444-444d-a0b5-9c841236f46f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:31.721819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 567/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:37:31 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:33.754883 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b33a8f-838d-4918-b489-5bbb15d1dbbd None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:33.755433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b33a8f-838d-4918-b489-5bbb15d1dbbd None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:33.755433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b33a8f-838d-4918-b489-5bbb15d1dbbd None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:33.756855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 572/1139] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:33.762029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302bcf2-d4d4-4453-bce3-14db6572e632 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:33.762439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302bcf2-d4d4-4453-bce3-14db6572e632 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:33.762585 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302bcf2-d4d4-4453-bce3-14db6572e632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:33.780088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d302bcf2-d4d4-4453-bce3-14db6572e632 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:33.781573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d302bcf2-d4d4-4453-bce3-14db6572e632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:33.811243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:33.848871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 568/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:33.872747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b33a8f-838d-4918-b489-5bbb15d1dbbd None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:33.884889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:33.972310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b33a8f-838d-4918-b489-5bbb15d1dbbd None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:34.006013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3cebb-fe6e-40e6-b087-e3e422aba033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:34.006723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3cebb-fe6e-40e6-b087-e3e422aba033 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:34.007002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3cebb-fe6e-40e6-b087-e3e422aba033 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:34.007272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3cebb-fe6e-40e6-b087-e3e422aba033 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:34.017486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3cebb-fe6e-40e6-b087-e3e422aba033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-39c3cebb-fe6e-40e6-b087-e3e422aba033 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:34.025219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:34.026376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:34.027277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 573/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:37:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:34.032960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302bcf2-d4d4-4453-bce3-14db6572e632 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:34.133314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302bcf2-d4d4-4453-bce3-14db6572e632 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:34.166723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c611135-4577-4af7-9d71-e37f674dc40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:34.167382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c611135-4577-4af7-9d71-e37f674dc40b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:34.167665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c611135-4577-4af7-9d71-e37f674dc40b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:34.167942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c611135-4577-4af7-9d71-e37f674dc40b None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:34.168713 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c611135-4577-4af7-9d71-e37f674dc40b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:34.169022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c611135-4577-4af7-9d71-e37f674dc40b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:34.170723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c611135-4577-4af7-9d71-e37f674dc40b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:34.178539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 569/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:37:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:34.184454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3cebb-fe6e-40e6-b087-e3e422aba033 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:34.253991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:34.286947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3cebb-fe6e-40e6-b087-e3e422aba033 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:34.319385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fb46b-5c73-42c3-8a3b-5f09e93b34f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:34.320237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fb46b-5c73-42c3-8a3b-5f09e93b34f0 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:34.320521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fb46b-5c73-42c3-8a3b-5f09e93b34f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:34.320827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fb46b-5c73-42c3-8a3b-5f09e93b34f0 None admin] PATH_INFO: `/v3/users/e9ba72f4d9844acb82fb028ea85cb072` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:34.334395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fb46b-5c73-42c3-8a3b-5f09e93b34f0 None admin] RBAC: Authorizing `identity:get_user(user_id=e9ba72f4d9844acb82fb028ea85cb072)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:34.335825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fb46b-5c73-42c3-8a3b-5f09e93b34f0 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:34.337497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 574/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:34 2022] GET /identity/v3/users/e9ba72f4d9844acb82fb028ea85cb072 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:34.344264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c611135-4577-4af7-9d71-e37f674dc40b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:34.381407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:34.456321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c611135-4577-4af7-9d71-e37f674dc40b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:34.488467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fe6db-9410-4291-b1f2-e5deeecd8ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:34.489142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fe6db-9410-4291-b1f2-e5deeecd8ce6 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:34.489428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fe6db-9410-4291-b1f2-e5deeecd8ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:34.489707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346fe6db-9410-4291-b1f2-e5deeecd8ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:34.555252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fe6db-9410-4291-b1f2-e5deeecd8ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:34.556654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346fe6db-9410-4291-b1f2-e5deeecd8ce6 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:34.585337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:34.661134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 570/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:37:34.667036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fb46b-5c73-42c3-8a3b-5f09e93b34f0 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:34.795893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fb46b-5c73-42c3-8a3b-5f09e93b34f0 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:34.821645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:34.829823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0cca3-081f-4c6b-9647-f265fef5d6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:34.830676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0cca3-081f-4c6b-9647-f265fef5d6f0 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:34.830960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0cca3-081f-4c6b-9647-f265fef5d6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:34.831238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0cca3-081f-4c6b-9647-f265fef5d6f0 None admin] PATH_INFO: `/v3/projects/8a1e09b036d94193b124a0d6b8ec24fb` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:34.832412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0cca3-081f-4c6b-9647-f265fef5d6f0 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1e09b036d94193b124a0d6b8ec24fb)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:34.833876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0cca3-081f-4c6b-9647-f265fef5d6f0 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:34.835570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 575/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:34 2022] GET /identity/v3/projects/8a1e09b036d94193b124a0d6b8ec24fb => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:34.841628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fe6db-9410-4291-b1f2-e5deeecd8ce6 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:34.938043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346fe6db-9410-4291-b1f2-e5deeecd8ce6 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:34.966126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:34.972535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52598eb2-37d4-4d12-a1ff-b9eb6729daab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:34.973287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52598eb2-37d4-4d12-a1ff-b9eb6729daab None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:34.973564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52598eb2-37d4-4d12-a1ff-b9eb6729daab None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:34.973835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52598eb2-37d4-4d12-a1ff-b9eb6729daab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:34.974656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52598eb2-37d4-4d12-a1ff-b9eb6729daab None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=8a1e09b036d94193b124a0d6b8ec24fb, user.id=e9ba72f4d9844acb82fb028ea85cb072) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:34.974965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52598eb2-37d4-4d12-a1ff-b9eb6729daab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:34.976377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52598eb2-37d4-4d12-a1ff-b9eb6729daab None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:34.981736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 571/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:37:34 2022] GET /identity/v3/role_assignments?user.id=e9ba72f4d9844acb82fb028ea85cb072&scope.project.id=8a1e09b036d94193b124a0d6b8ec24fb&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:36.968855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73640db0-6eea-4abe-92fb-00ed8eef382d None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:36.969305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73640db0-6eea-4abe-92fb-00ed8eef382d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:36.969437 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73640db0-6eea-4abe-92fb-00ed8eef382d None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:36.970759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 576/1147] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:36.975985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d90512-a7bf-4321-9151-c77eec8e5492 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:36.976377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d90512-a7bf-4321-9151-c77eec8e5492 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:36.976529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d90512-a7bf-4321-9151-c77eec8e5492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:36.993480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a6d90512-a7bf-4321-9151-c77eec8e5492 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:36.994640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a6d90512-a7bf-4321-9151-c77eec8e5492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:37.058708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 572/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:37.081616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73640db0-6eea-4abe-92fb-00ed8eef382d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:37.179180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73640db0-6eea-4abe-92fb-00ed8eef382d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:37.211715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174b9aa-3ede-4010-8fd6-b4e00c64ece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:37.212426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174b9aa-3ede-4010-8fd6-b4e00c64ece2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:37.212745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174b9aa-3ede-4010-8fd6-b4e00c64ece2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:37.213020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174b9aa-3ede-4010-8fd6-b4e00c64ece2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:37.222884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174b9aa-3ede-4010-8fd6-b4e00c64ece2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-e174b9aa-3ede-4010-8fd6-b4e00c64ece2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:37.230386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:37.231448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:37.233046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 577/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:37:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:37.238461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d90512-a7bf-4321-9151-c77eec8e5492 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:37.332600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d90512-a7bf-4321-9151-c77eec8e5492 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:37.355086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:37.366638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de577663-145d-4432-817f-a931eb294452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:37.367293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de577663-145d-4432-817f-a931eb294452 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:37.367574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de577663-145d-4432-817f-a931eb294452 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:37.367848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de577663-145d-4432-817f-a931eb294452 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:37.368605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de577663-145d-4432-817f-a931eb294452 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:37.368914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de577663-145d-4432-817f-a931eb294452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:37.370261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de577663-145d-4432-817f-a931eb294452 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:37.378079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 573/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:37:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:37.383743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174b9aa-3ede-4010-8fd6-b4e00c64ece2 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:37.439654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:37.482256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174b9aa-3ede-4010-8fd6-b4e00c64ece2 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:37.515158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713a7a3-3fcf-43f7-9956-b9f75297c5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:37.516013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713a7a3-3fcf-43f7-9956-b9f75297c5cf None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:37.516295 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713a7a3-3fcf-43f7-9956-b9f75297c5cf None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:37.516571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713a7a3-3fcf-43f7-9956-b9f75297c5cf None admin] PATH_INFO: `/v3/users/e9ba72f4d9844acb82fb028ea85cb072` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:37.517767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713a7a3-3fcf-43f7-9956-b9f75297c5cf None admin] RBAC: Authorizing `identity:get_user(user_id=e9ba72f4d9844acb82fb028ea85cb072)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:37.519199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713a7a3-3fcf-43f7-9956-b9f75297c5cf None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:37.520838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 578/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:37 2022] GET /identity/v3/users/e9ba72f4d9844acb82fb028ea85cb072 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:37.526682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de577663-145d-4432-817f-a931eb294452 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:37.624564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de577663-145d-4432-817f-a931eb294452 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:37.657363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a277d-1e46-4af8-8620-dc2789d9709e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:37.658211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a277d-1e46-4af8-8620-dc2789d9709e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:37.658495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a277d-1e46-4af8-8620-dc2789d9709e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:37.658772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a277d-1e46-4af8-8620-dc2789d9709e None admin] PATH_INFO: `/v3/projects/8a1e09b036d94193b124a0d6b8ec24fb` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:37.659939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a277d-1e46-4af8-8620-dc2789d9709e None admin] RBAC: Authorizing `identity:get_project(project_id=8a1e09b036d94193b124a0d6b8ec24fb)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:37.661389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a277d-1e46-4af8-8620-dc2789d9709e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:37.663063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 574/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:37 2022] GET /identity/v3/projects/8a1e09b036d94193b124a0d6b8ec24fb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:37.668976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713a7a3-3fcf-43f7-9956-b9f75297c5cf None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:37.768452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713a7a3-3fcf-43f7-9956-b9f75297c5cf None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:37.793461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:37.804790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a132f-2fef-4aa3-b382-416f15bec56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:37.805676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a132f-2fef-4aa3-b382-416f15bec56e None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:37.805960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a132f-2fef-4aa3-b382-416f15bec56e None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:37.806238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a132f-2fef-4aa3-b382-416f15bec56e None admin] PATH_INFO: `/v3/projects/8a1e09b036d94193b124a0d6b8ec24fb/users/e9ba72f4d9844acb82fb028ea85cb072/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:37.808186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a132f-2fef-4aa3-b382-416f15bec56e None admin] RBAC: Authorizing `identity:create_grant(project_id=8a1e09b036d94193b124a0d6b8ec24fb, user_id=e9ba72f4d9844acb82fb028ea85cb072, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:37.809658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a132f-2fef-4aa3-b382-416f15bec56e None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:37.822402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 579/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:37:37 2022] PUT /identity/v3/projects/8a1e09b036d94193b124a0d6b8ec24fb/users/e9ba72f4d9844acb82fb028ea85cb072/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:37:39.840280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30b9e0-1de8-4a9e-a0d3-b3fcd694ff80 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:39.840885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30b9e0-1de8-4a9e-a0d3-b3fcd694ff80 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:39.840885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30b9e0-1de8-4a9e-a0d3-b3fcd694ff80 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:39.842353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 575/1154] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:39.847485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bfd6b-74f5-420e-bbe4-5c76a73da699 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:39.847901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bfd6b-74f5-420e-bbe4-5c76a73da699 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:39.848046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bfd6b-74f5-420e-bbe4-5c76a73da699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:39.864529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c63bfd6b-74f5-420e-bbe4-5c76a73da699 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:39.865770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c63bfd6b-74f5-420e-bbe4-5c76a73da699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:39.930151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 580/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:39.953922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30b9e0-1de8-4a9e-a0d3-b3fcd694ff80 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:40.009088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:40.054401 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30b9e0-1de8-4a9e-a0d3-b3fcd694ff80 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:40.086536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0682-4505-4f62-8ea8-edbdec393e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:40.087313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0682-4505-4f62-8ea8-edbdec393e10 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:40.087596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0682-4505-4f62-8ea8-edbdec393e10 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:40.087876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0682-4505-4f62-8ea8-edbdec393e10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:40.098699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b0682-4505-4f62-8ea8-edbdec393e10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-145b0682-4505-4f62-8ea8-edbdec393e10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:40.106191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:40.107288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:40.108412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 576/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:37:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:40.113644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bfd6b-74f5-420e-bbe4-5c76a73da699 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:40.209419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bfd6b-74f5-420e-bbe4-5c76a73da699 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:40.223113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:40.243960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93e38-0c98-4210-98e7-90878bf3d92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:40.244639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93e38-0c98-4210-98e7-90878bf3d92d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:40.244928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93e38-0c98-4210-98e7-90878bf3d92d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:40.245206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93e38-0c98-4210-98e7-90878bf3d92d None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:40.245947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93e38-0c98-4210-98e7-90878bf3d92d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:40.246257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93e38-0c98-4210-98e7-90878bf3d92d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:40.247625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93e38-0c98-4210-98e7-90878bf3d92d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:40.256216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 581/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:37:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:40.261813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0682-4505-4f62-8ea8-edbdec393e10 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:40.359303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0682-4505-4f62-8ea8-edbdec393e10 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:40.373578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:40.393250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80857743-ff18-4b7c-af2a-7ee6f34bec9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:40.394107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80857743-ff18-4b7c-af2a-7ee6f34bec9b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:40.394395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80857743-ff18-4b7c-af2a-7ee6f34bec9b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:40.394669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80857743-ff18-4b7c-af2a-7ee6f34bec9b None admin] PATH_INFO: `/v3/users/e9ba72f4d9844acb82fb028ea85cb072` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:40.395852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80857743-ff18-4b7c-af2a-7ee6f34bec9b None admin] RBAC: Authorizing `identity:get_user(user_id=e9ba72f4d9844acb82fb028ea85cb072)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:40.397562 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80857743-ff18-4b7c-af2a-7ee6f34bec9b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:40.399145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 577/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:40 2022] GET /identity/v3/users/e9ba72f4d9844acb82fb028ea85cb072 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:40.405484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93e38-0c98-4210-98e7-90878bf3d92d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:40.503856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93e38-0c98-4210-98e7-90878bf3d92d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:40.536390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad9cc6-df2d-49cd-bef6-96e984763ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:40.537057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad9cc6-df2d-49cd-bef6-96e984763ab6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:40.537342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad9cc6-df2d-49cd-bef6-96e984763ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:40.537621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad9cc6-df2d-49cd-bef6-96e984763ab6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:40.580255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:40.602666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad9cc6-df2d-49cd-bef6-96e984763ab6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:40.604111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad9cc6-df2d-49cd-bef6-96e984763ab6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:40.700713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 582/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 23:37:40.706476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80857743-ff18-4b7c-af2a-7ee6f34bec9b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:40.803762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80857743-ff18-4b7c-af2a-7ee6f34bec9b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:40.836420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11d1d3-20ea-4b98-aa07-350e5ef8c610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:40.837298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11d1d3-20ea-4b98-aa07-350e5ef8c610 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:40.837580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11d1d3-20ea-4b98-aa07-350e5ef8c610 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:40.837858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11d1d3-20ea-4b98-aa07-350e5ef8c610 None admin] PATH_INFO: `/v3/projects/8a1e09b036d94193b124a0d6b8ec24fb` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:40.839028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11d1d3-20ea-4b98-aa07-350e5ef8c610 None admin] RBAC: Authorizing `identity:get_project(project_id=8a1e09b036d94193b124a0d6b8ec24fb)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:40.840452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a11d1d3-20ea-4b98-aa07-350e5ef8c610 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:40.842846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 578/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:40 2022] GET /identity/v3/projects/8a1e09b036d94193b124a0d6b8ec24fb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:40.849847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad9cc6-df2d-49cd-bef6-96e984763ab6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:40.926546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:40.949163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad9cc6-df2d-49cd-bef6-96e984763ab6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:40.983262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb77bea-5655-4987-aaba-270f3dd21200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:40.983907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb77bea-5655-4987-aaba-270f3dd21200 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:40.984194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb77bea-5655-4987-aaba-270f3dd21200 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:40.984474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb77bea-5655-4987-aaba-270f3dd21200 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:40.985330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb77bea-5655-4987-aaba-270f3dd21200 None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=8a1e09b036d94193b124a0d6b8ec24fb, user.id=e9ba72f4d9844acb82fb028ea85cb072) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:40.985655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb77bea-5655-4987-aaba-270f3dd21200 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:40.987083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb77bea-5655-4987-aaba-270f3dd21200 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:40.992801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 583/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:37:40 2022] GET /identity/v3/role_assignments?user.id=e9ba72f4d9844acb82fb028ea85cb072&scope.project.id=8a1e09b036d94193b124a0d6b8ec24fb&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:43.031473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07005092-38d6-442e-bcda-e1c9e17379d6 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:43.031950 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07005092-38d6-442e-bcda-e1c9e17379d6 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:43.032060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07005092-38d6-442e-bcda-e1c9e17379d6 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:43.033396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 579/1162] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:43.038667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6882f-87ad-46cc-87ff-c9090fb80a41 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:43.039081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6882f-87ad-46cc-87ff-c9090fb80a41 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:43.039230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6882f-87ad-46cc-87ff-c9090fb80a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:43.056025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-dba6882f-87ad-46cc-87ff-c9090fb80a41 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:43.057246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-dba6882f-87ad-46cc-87ff-c9090fb80a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:43.138814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 584/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:43.161206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07005092-38d6-442e-bcda-e1c9e17379d6 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:43.210854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:43.259134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07005092-38d6-442e-bcda-e1c9e17379d6 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:43.291238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c29da2-cef0-48d9-8a86-f084b92fcd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:43.291872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c29da2-cef0-48d9-8a86-f084b92fcd4d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:43.292152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c29da2-cef0-48d9-8a86-f084b92fcd4d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:43.292425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c29da2-cef0-48d9-8a86-f084b92fcd4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:43.356225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c29da2-cef0-48d9-8a86-f084b92fcd4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:43.357717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c29da2-cef0-48d9-8a86-f084b92fcd4d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:43.403501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:43.469587 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 580/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:37:43.475510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6882f-87ad-46cc-87ff-c9090fb80a41 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:43.542859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:43.572783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6882f-87ad-46cc-87ff-c9090fb80a41 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:43.605056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b0543c-4ec1-48ee-9f40-f44e3f485be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:43.605834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0543c-4ec1-48ee-9f40-f44e3f485be9 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:43.606112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0543c-4ec1-48ee-9f40-f44e3f485be9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:43.606385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b0543c-4ec1-48ee-9f40-f44e3f485be9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:43.612787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0543c-4ec1-48ee-9f40-f44e3f485be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:43.614216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b0543c-4ec1-48ee-9f40-f44e3f485be9 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-28b0543c-4ec1-48ee-9f40-f44e3f485be9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:43.628447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:43.629678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:43.630723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 585/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:37:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:43.635608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c29da2-cef0-48d9-8a86-f084b92fcd4d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:43.729893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c29da2-cef0-48d9-8a86-f084b92fcd4d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:43.773558 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dda236-0a80-4483-bc57-54a3a0f5cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:43.774212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dda236-0a80-4483-bc57-54a3a0f5cb9d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:43.774490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dda236-0a80-4483-bc57-54a3a0f5cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:43.774763 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dda236-0a80-4483-bc57-54a3a0f5cb9d None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:43.775477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dda236-0a80-4483-bc57-54a3a0f5cb9d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:43.775780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dda236-0a80-4483-bc57-54a3a0f5cb9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:43.777162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dda236-0a80-4483-bc57-54a3a0f5cb9d None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:43.784104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 581/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Fri Jun 24 23:37:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:45.774357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110d42b-497a-4ca9-a059-eb287d90382f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:45.774887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110d42b-497a-4ca9-a059-eb287d90382f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:45.774998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110d42b-497a-4ca9-a059-eb287d90382f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:45.776290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 586/1167] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:45.781864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd470b2-0746-474b-ac6f-c421b4f1fd5f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:45.782240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd470b2-0746-474b-ac6f-c421b4f1fd5f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:45.782423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd470b2-0746-474b-ac6f-c421b4f1fd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:45.799173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0bd470b2-0746-474b-ac6f-c421b4f1fd5f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:45.800339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0bd470b2-0746-474b-ac6f-c421b4f1fd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:45.809179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:45.867707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 582/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:45.892776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110d42b-497a-4ca9-a059-eb287d90382f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:45.992872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110d42b-497a-4ca9-a059-eb287d90382f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:46.020000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:46.026317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9a63d-7256-4c7c-92d2-6602a895e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:46.026911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9a63d-7256-4c7c-92d2-6602a895e399 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:46.027193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9a63d-7256-4c7c-92d2-6602a895e399 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:46.027467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9a63d-7256-4c7c-92d2-6602a895e399 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:46.028212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9a63d-7256-4c7c-92d2-6602a895e399 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:46.029703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9a63d-7256-4c7c-92d2-6602a895e399 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:46.044359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 587/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Fri Jun 24 23:37:45 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:48.086994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda3174-a8d5-4e2c-a99a-9076e6f14b8b None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:48.087960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda3174-a8d5-4e2c-a99a-9076e6f14b8b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:48.088428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda3174-a8d5-4e2c-a99a-9076e6f14b8b None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:48.089821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 583/1170] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:48.095612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9abcc7-c9da-4def-adeb-f6754a1bc2b4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:48.096072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9abcc7-c9da-4def-adeb-f6754a1bc2b4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:48.096172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9abcc7-c9da-4def-adeb-f6754a1bc2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:48.113326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3e9abcc7-c9da-4def-adeb-f6754a1bc2b4 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:48.114519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3e9abcc7-c9da-4def-adeb-f6754a1bc2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:48.182965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 588/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:48.207514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda3174-a8d5-4e2c-a99a-9076e6f14b8b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:48.307363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda3174-a8d5-4e2c-a99a-9076e6f14b8b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:48.339943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5234780-c1fe-458e-b53c-e82b24326b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:48.340822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234780-c1fe-458e-b53c-e82b24326b14 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:48.341105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234780-c1fe-458e-b53c-e82b24326b14 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:48.341998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5234780-c1fe-458e-b53c-e82b24326b14 None admin] PATH_INFO: `/v3/domains/d822f2d92d4b4058b765bd9d3414edee` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:48.352213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234780-c1fe-458e-b53c-e82b24326b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d822f2d92d4b4058b765bd9d3414edee)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:48.353673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5234780-c1fe-458e-b53c-e82b24326b14 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:48.355282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 584/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Fri Jun 24 23:37:48 2022] GET /identity/v3/domains/d822f2d92d4b4058b765bd9d3414edee => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:48.361090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9abcc7-c9da-4def-adeb-f6754a1bc2b4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:48.456129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9abcc7-c9da-4def-adeb-f6754a1bc2b4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:48.475685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:48.491293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a74fa9-92b0-488c-afa7-6dfc16e753c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:48.491941 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a74fa9-92b0-488c-afa7-6dfc16e753c4 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:48.492222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a74fa9-92b0-488c-afa7-6dfc16e753c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:48.492496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a74fa9-92b0-488c-afa7-6dfc16e753c4 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:48.493237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a74fa9-92b0-488c-afa7-6dfc16e753c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:48.494664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a74fa9-92b0-488c-afa7-6dfc16e753c4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:48.518562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 589/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:37:48 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:50.583839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6f682-eaa4-477f-820f-2470c4d13799 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:50.584397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6f682-eaa4-477f-820f-2470c4d13799 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:50.584397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6f682-eaa4-477f-820f-2470c4d13799 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:50.585772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 585/1174] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:50.590877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb228e79-8a17-48e5-8546-e04e012d7e3e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:50.591331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb228e79-8a17-48e5-8546-e04e012d7e3e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:50.591490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb228e79-8a17-48e5-8546-e04e012d7e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:50.608689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-eb228e79-8a17-48e5-8546-e04e012d7e3e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:50.609863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-eb228e79-8a17-48e5-8546-e04e012d7e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:50.674062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 590/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:50.698267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6f682-eaa4-477f-820f-2470c4d13799 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:50.702593 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:50.798001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6f682-eaa4-477f-820f-2470c4d13799 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:50.830256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32878d83-7a8d-4762-b5cb-5aee20e5d57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:50.831108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32878d83-7a8d-4762-b5cb-5aee20e5d57f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:50.831387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32878d83-7a8d-4762-b5cb-5aee20e5d57f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:50.831662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32878d83-7a8d-4762-b5cb-5aee20e5d57f None admin] PATH_INFO: `/v3/domains/d822f2d92d4b4058b765bd9d3414edee` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:50.832850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32878d83-7a8d-4762-b5cb-5aee20e5d57f None admin] RBAC: Authorizing `identity:get_domain(domain_id=d822f2d92d4b4058b765bd9d3414edee)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:50.834323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32878d83-7a8d-4762-b5cb-5aee20e5d57f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:50.835866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 586/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Fri Jun 24 23:37:50 2022] GET /identity/v3/domains/d822f2d92d4b4058b765bd9d3414edee => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:50.842102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb228e79-8a17-48e5-8546-e04e012d7e3e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:50.936523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb228e79-8a17-48e5-8546-e04e012d7e3e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:50.938260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:50.972489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3af57c-5c97-489f-aefb-be02e2e2f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:50.973198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3af57c-5c97-489f-aefb-be02e2e2f487 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:50.973498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3af57c-5c97-489f-aefb-be02e2e2f487 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:50.973766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3af57c-5c97-489f-aefb-be02e2e2f487 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:50.974491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3af57c-5c97-489f-aefb-be02e2e2f487 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:50.975933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3af57c-5c97-489f-aefb-be02e2e2f487 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:50.995113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 591/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:37:50 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:37:53.095134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d59c0e-52f5-47ee-8d2e-f349b9696914 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:53.095657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d59c0e-52f5-47ee-8d2e-f349b9696914 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:53.095760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d59c0e-52f5-47ee-8d2e-f349b9696914 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:53.097089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 587/1178] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:53.102162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135d07a-84ac-424a-906c-959d183951dd None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:53.102573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135d07a-84ac-424a-906c-959d183951dd None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:53.102724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135d07a-84ac-424a-906c-959d183951dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:53.123549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9135d07a-84ac-424a-906c-959d183951dd None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:53.124780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9135d07a-84ac-424a-906c-959d183951dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:53.193499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 592/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:37:53.218187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d59c0e-52f5-47ee-8d2e-f349b9696914 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:53.286475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:53.316695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d59c0e-52f5-47ee-8d2e-f349b9696914 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:53.349843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b362f73-7e66-40d9-825b-6d653eba223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:53.350624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b362f73-7e66-40d9-825b-6d653eba223e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:53.350904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b362f73-7e66-40d9-825b-6d653eba223e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:53.351174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b362f73-7e66-40d9-825b-6d653eba223e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:53.362492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b362f73-7e66-40d9-825b-6d653eba223e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-4b362f73-7e66-40d9-825b-6d653eba223e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:53.370027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:53.371177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:53.372079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 588/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:37:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:53.377575 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135d07a-84ac-424a-906c-959d183951dd None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:53.462651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:53.475932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135d07a-84ac-424a-906c-959d183951dd None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:53.510129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75243e1d-77c9-4c4e-8d3b-2b596f87684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:53.510786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243e1d-77c9-4c4e-8d3b-2b596f87684b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:53.511068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243e1d-77c9-4c4e-8d3b-2b596f87684b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:53.511346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75243e1d-77c9-4c4e-8d3b-2b596f87684b None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:53.512087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75243e1d-77c9-4c4e-8d3b-2b596f87684b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:53.512396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75243e1d-77c9-4c4e-8d3b-2b596f87684b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:53.514782 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75243e1d-77c9-4c4e-8d3b-2b596f87684b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:53.522771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 593/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:37:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:53.529182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b362f73-7e66-40d9-825b-6d653eba223e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:53.626674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b362f73-7e66-40d9-825b-6d653eba223e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:53.657300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:53.662482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bdc14-58e0-48d1-adba-1009bec90b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:53.663347 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bdc14-58e0-48d1-adba-1009bec90b43 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:53.663631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bdc14-58e0-48d1-adba-1009bec90b43 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:53.663906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bdc14-58e0-48d1-adba-1009bec90b43 None admin] PATH_INFO: `/v3/users/86000e25452a475a8d688a21f08b461a` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:53.677236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147bdc14-58e0-48d1-adba-1009bec90b43 None admin] RBAC: Authorizing `identity:get_user(user_id=86000e25452a475a8d688a21f08b461a)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:53.678660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147bdc14-58e0-48d1-adba-1009bec90b43 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:53.680271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 589/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:53 2022] GET /identity/v3/users/86000e25452a475a8d688a21f08b461a => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:53.686380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75243e1d-77c9-4c4e-8d3b-2b596f87684b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:53.789932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75243e1d-77c9-4c4e-8d3b-2b596f87684b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:53.823885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8ff02-0b11-4be3-a171-528dbbe79b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:53.824526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8ff02-0b11-4be3-a171-528dbbe79b35 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:53.824837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8ff02-0b11-4be3-a171-528dbbe79b35 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:53.825113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8ff02-0b11-4be3-a171-528dbbe79b35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:53.849361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:53.898592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8ff02-0b11-4be3-a171-528dbbe79b35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:53.900038 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8ff02-0b11-4be3-a171-528dbbe79b35 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:53.999219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 594/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:37:54.005134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bdc14-58e0-48d1-adba-1009bec90b43 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:54.109164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bdc14-58e0-48d1-adba-1009bec90b43 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:54.142317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221daf6a-bfef-404e-a149-fc6c9f37f7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:54.143166 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221daf6a-bfef-404e-a149-fc6c9f37f7dd None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:54.143446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221daf6a-bfef-404e-a149-fc6c9f37f7dd None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:54.143730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221daf6a-bfef-404e-a149-fc6c9f37f7dd None admin] PATH_INFO: `/v3/projects/287dfc58a50e49b89372c933399ffe3c` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:54.144938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221daf6a-bfef-404e-a149-fc6c9f37f7dd None admin] RBAC: Authorizing `identity:get_project(project_id=287dfc58a50e49b89372c933399ffe3c)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:54.146361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221daf6a-bfef-404e-a149-fc6c9f37f7dd None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:54.148278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 590/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:54 2022] GET /identity/v3/projects/287dfc58a50e49b89372c933399ffe3c => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:54.154391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8ff02-0b11-4be3-a171-528dbbe79b35 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:54.202401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:54.250465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8ff02-0b11-4be3-a171-528dbbe79b35 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:54.283053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60838b63-6104-41d3-9bbd-188db8137068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:54.283692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60838b63-6104-41d3-9bbd-188db8137068 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:54.283972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60838b63-6104-41d3-9bbd-188db8137068 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:54.284245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60838b63-6104-41d3-9bbd-188db8137068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:54.285087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60838b63-6104-41d3-9bbd-188db8137068 None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=287dfc58a50e49b89372c933399ffe3c, user.id=86000e25452a475a8d688a21f08b461a) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:54.285398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60838b63-6104-41d3-9bbd-188db8137068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:54.286805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60838b63-6104-41d3-9bbd-188db8137068 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:54.292031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 595/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:37:54 2022] GET /identity/v3/role_assignments?user.id=86000e25452a475a8d688a21f08b461a&scope.project.id=287dfc58a50e49b89372c933399ffe3c&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:37:56.262630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1c60a-4e5b-4f7a-89da-7e6d4914777f None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:56.263128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1c60a-4e5b-4f7a-89da-7e6d4914777f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:56.263128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1c60a-4e5b-4f7a-89da-7e6d4914777f None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:56.264490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 591/1186] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:56.269548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98564a08-8357-468b-a139-12bc36d010e7 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:56.269962 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98564a08-8357-468b-a139-12bc36d010e7 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:56.270117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98564a08-8357-468b-a139-12bc36d010e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:56.287241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-98564a08-8357-468b-a139-12bc36d010e7 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:56.288432 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-98564a08-8357-468b-a139-12bc36d010e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:56.353490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 596/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:56.376568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1c60a-4e5b-4f7a-89da-7e6d4914777f None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:56.435462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:56.476181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1c60a-4e5b-4f7a-89da-7e6d4914777f None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:56.509175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90901ccf-0b6d-4adf-b07a-0c6436f7086d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:56.509953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90901ccf-0b6d-4adf-b07a-0c6436f7086d None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:56.510234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90901ccf-0b6d-4adf-b07a-0c6436f7086d None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:56.510508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90901ccf-0b6d-4adf-b07a-0c6436f7086d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:56.521073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90901ccf-0b6d-4adf-b07a-0c6436f7086d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-90901ccf-0b6d-4adf-b07a-0c6436f7086d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:56.529581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:56.530678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:37:56.531675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 592/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:37:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:56.536925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98564a08-8357-468b-a139-12bc36d010e7 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:56.581272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:56.633228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98564a08-8357-468b-a139-12bc36d010e7 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:56.666338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62273a42-bff1-48b3-8367-5d9ec28334e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:56.666991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62273a42-bff1-48b3-8367-5d9ec28334e1 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:56.667273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62273a42-bff1-48b3-8367-5d9ec28334e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:56.667545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62273a42-bff1-48b3-8367-5d9ec28334e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:56.668279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62273a42-bff1-48b3-8367-5d9ec28334e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:56.668600 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62273a42-bff1-48b3-8367-5d9ec28334e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:56.670254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62273a42-bff1-48b3-8367-5d9ec28334e1 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:56.677960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 597/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:37:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:56.683631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90901ccf-0b6d-4adf-b07a-0c6436f7086d None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:56.781547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90901ccf-0b6d-4adf-b07a-0c6436f7086d None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:56.797588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:56.814071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb91ebd-b8a7-4f6b-943b-11602fd06ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:56.814930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb91ebd-b8a7-4f6b-943b-11602fd06ae1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:56.815210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb91ebd-b8a7-4f6b-943b-11602fd06ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:56.815481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb91ebd-b8a7-4f6b-943b-11602fd06ae1 None admin] PATH_INFO: `/v3/users/86000e25452a475a8d688a21f08b461a` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:56.816686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb91ebd-b8a7-4f6b-943b-11602fd06ae1 None admin] RBAC: Authorizing `identity:get_user(user_id=86000e25452a475a8d688a21f08b461a)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:56.818120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb91ebd-b8a7-4f6b-943b-11602fd06ae1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:56.819714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 593/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:56 2022] GET /identity/v3/users/86000e25452a475a8d688a21f08b461a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:56.825932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62273a42-bff1-48b3-8367-5d9ec28334e1 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:56.922147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62273a42-bff1-48b3-8367-5d9ec28334e1 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:56.928692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:56.956168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f508faa-27f7-4cd2-ad48-5367fdb19d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:56.957121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f508faa-27f7-4cd2-ad48-5367fdb19d39 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:56.957409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f508faa-27f7-4cd2-ad48-5367fdb19d39 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:56.957688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f508faa-27f7-4cd2-ad48-5367fdb19d39 None admin] PATH_INFO: `/v3/projects/287dfc58a50e49b89372c933399ffe3c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:56.958875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f508faa-27f7-4cd2-ad48-5367fdb19d39 None admin] RBAC: Authorizing `identity:get_project(project_id=287dfc58a50e49b89372c933399ffe3c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:56.960302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f508faa-27f7-4cd2-ad48-5367fdb19d39 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:56.962041 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 598/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:37:56 2022] GET /identity/v3/projects/287dfc58a50e49b89372c933399ffe3c => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:56.967877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb91ebd-b8a7-4f6b-943b-11602fd06ae1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:57.063945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb91ebd-b8a7-4f6b-943b-11602fd06ae1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:57.097372 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf495c33-6811-4d7b-b475-8fe81a877f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:57.098252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf495c33-6811-4d7b-b475-8fe81a877f27 None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:57.098535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf495c33-6811-4d7b-b475-8fe81a877f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:57.098812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf495c33-6811-4d7b-b475-8fe81a877f27 None admin] PATH_INFO: `/v3/projects/287dfc58a50e49b89372c933399ffe3c/users/86000e25452a475a8d688a21f08b461a/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:57.100784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf495c33-6811-4d7b-b475-8fe81a877f27 None admin] RBAC: Authorizing `identity:create_grant(project_id=287dfc58a50e49b89372c933399ffe3c, user_id=86000e25452a475a8d688a21f08b461a, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:57.102205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf495c33-6811-4d7b-b475-8fe81a877f27 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:57.115323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 594/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:37:56 2022] PUT /identity/v3/projects/287dfc58a50e49b89372c933399ffe3c/users/86000e25452a475a8d688a21f08b461a/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:37:59.133487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff347a09-1cb2-4370-9bbf-e6375f4ceece None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:59.134000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff347a09-1cb2-4370-9bbf-e6375f4ceece None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:59.134105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff347a09-1cb2-4370-9bbf-e6375f4ceece None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:59.135405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 599/1193] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:37:59.141225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba4335-7d90-4269-9fa9-a760aafcd09d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:59.141617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba4335-7d90-4269-9fa9-a760aafcd09d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:59.141791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba4335-7d90-4269-9fa9-a760aafcd09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:59.158490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-aaba4335-7d90-4269-9fa9-a760aafcd09d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:37:59.159675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-aaba4335-7d90-4269-9fa9-a760aafcd09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:37:59.201055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:59.225513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 595/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:37:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:37:59.249120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff347a09-1cb2-4370-9bbf-e6375f4ceece None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:59.350953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff347a09-1cb2-4370-9bbf-e6375f4ceece None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:59.383971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761a124-d4ad-41fd-af6b-d873d39d22cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:59.384782 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761a124-d4ad-41fd-af6b-d873d39d22cd None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:59.385070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761a124-d4ad-41fd-af6b-d873d39d22cd None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:59.385349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761a124-d4ad-41fd-af6b-d873d39d22cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:59.396124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761a124-d4ad-41fd-af6b-d873d39d22cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-4761a124-d4ad-41fd-af6b-d873d39d22cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:37:59.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:37:59.405032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:37:59.406053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 600/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:37:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:37:59.411072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba4335-7d90-4269-9fa9-a760aafcd09d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:59.506453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba4335-7d90-4269-9fa9-a760aafcd09d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:59.538660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc1d43-41df-4df6-938f-ad93e334f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:59.539321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1d43-41df-4df6-938f-ad93e334f748 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:59.539605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1d43-41df-4df6-938f-ad93e334f748 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:59.539883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc1d43-41df-4df6-938f-ad93e334f748 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:59.540653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc1d43-41df-4df6-938f-ad93e334f748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:37:59.540962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc1d43-41df-4df6-938f-ad93e334f748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:59.542322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc1d43-41df-4df6-938f-ad93e334f748 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:59.550751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 596/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:37:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:59.556524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761a124-d4ad-41fd-af6b-d873d39d22cd None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:59.589687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:59.656503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761a124-d4ad-41fd-af6b-d873d39d22cd None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:59.690889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ad95c-b005-4a2b-82d7-2b67c48f79d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:59.691748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ad95c-b005-4a2b-82d7-2b67c48f79d4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:59.692032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ad95c-b005-4a2b-82d7-2b67c48f79d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:59.692308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ad95c-b005-4a2b-82d7-2b67c48f79d4 None admin] PATH_INFO: `/v3/users/86000e25452a475a8d688a21f08b461a` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:59.693516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ad95c-b005-4a2b-82d7-2b67c48f79d4 None admin] RBAC: Authorizing `identity:get_user(user_id=86000e25452a475a8d688a21f08b461a)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:59.694945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ad95c-b005-4a2b-82d7-2b67c48f79d4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:59.696546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 601/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Fri Jun 24 23:37:59 2022] GET /identity/v3/users/86000e25452a475a8d688a21f08b461a => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:37:59.702515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc1d43-41df-4df6-938f-ad93e334f748 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:37:59.741049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:37:59.798851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc1d43-41df-4df6-938f-ad93e334f748 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:37:59.832100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502c368-e1a9-4a6a-bd4c-81946ff87eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:37:59.832772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502c368-e1a9-4a6a-bd4c-81946ff87eb8 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:37:59.833056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502c368-e1a9-4a6a-bd4c-81946ff87eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:37:59.833332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502c368-e1a9-4a6a-bd4c-81946ff87eb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:37:59.904003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502c368-e1a9-4a6a-bd4c-81946ff87eb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:37:59.905449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502c368-e1a9-4a6a-bd4c-81946ff87eb8 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:37:59.941764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:38:00.001274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 597/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:37:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:38:00.007056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ad95c-b005-4a2b-82d7-2b67c48f79d4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:38:00.103749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ad95c-b005-4a2b-82d7-2b67c48f79d4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:38:00.105286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:38:00.138034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8c2b-3a63-45f4-a5de-bf556490c79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:38:00.138878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8c2b-3a63-45f4-a5de-bf556490c79b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:00.139164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8c2b-3a63-45f4-a5de-bf556490c79b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:00.139440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8c2b-3a63-45f4-a5de-bf556490c79b None admin] PATH_INFO: `/v3/projects/287dfc58a50e49b89372c933399ffe3c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:00.140642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d8c2b-3a63-45f4-a5de-bf556490c79b None admin] RBAC: Authorizing `identity:get_project(project_id=287dfc58a50e49b89372c933399ffe3c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:38:00.142075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d8c2b-3a63-45f4-a5de-bf556490c79b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:38:00.143735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 602/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Fri Jun 24 23:38:00 2022] GET /identity/v3/projects/287dfc58a50e49b89372c933399ffe3c => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:38:00.149949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502c368-e1a9-4a6a-bd4c-81946ff87eb8 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:38:00.245424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502c368-e1a9-4a6a-bd4c-81946ff87eb8 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:38:00.278900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b4afe-aa61-4956-abf3-4f51cec8261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:38:00.279528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b4afe-aa61-4956-abf3-4f51cec8261a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:00.279814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b4afe-aa61-4956-abf3-4f51cec8261a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:00.280091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b4afe-aa61-4956-abf3-4f51cec8261a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:00.280948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b4afe-aa61-4956-abf3-4f51cec8261a None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=287dfc58a50e49b89372c933399ffe3c, user.id=86000e25452a475a8d688a21f08b461a) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:38:00.281261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b4afe-aa61-4956-abf3-4f51cec8261a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:38:00.282661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b4afe-aa61-4956-abf3-4f51cec8261a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:38:00.288314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 598/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:38:00 2022] GET /identity/v3/role_assignments?user.id=86000e25452a475a8d688a21f08b461a&scope.project.id=287dfc58a50e49b89372c933399ffe3c&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:38:10.133877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=15 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:38:10.285879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:38:56.052244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33296437-eb86-49b4-8fd5-58f1f2c58561 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:56.052867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33296437-eb86-49b4-8fd5-58f1f2c58561 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:56.052867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33296437-eb86-49b4-8fd5-58f1f2c58561 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:56.054494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 603/1201] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:38:56.062794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8b78-ae51-4819-9ad4-82a991bd4769 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:56.063290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8b78-ae51-4819-9ad4-82a991bd4769 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:56.063493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8b78-ae51-4819-9ad4-82a991bd4769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:56.082593 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-605d8b78-ae51-4819-9ad4-82a991bd4769 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:38:56.083909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-605d8b78-ae51-4819-9ad4-82a991bd4769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:38:56.172348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 599/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:38:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:38:56.200824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33296437-eb86-49b4-8fd5-58f1f2c58561 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:38:56.332249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33296437-eb86-49b4-8fd5-58f1f2c58561 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:38:56.378518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235066a1-419b-4b33-83a4-6c0b41357fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:38:56.379773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235066a1-419b-4b33-83a4-6c0b41357fac None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:56.380156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235066a1-419b-4b33-83a4-6c0b41357fac None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:56.380451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235066a1-419b-4b33-83a4-6c0b41357fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:56.393829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235066a1-419b-4b33-83a4-6c0b41357fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:38:56.395537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235066a1-419b-4b33-83a4-6c0b41357fac None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-235066a1-419b-4b33-83a4-6c0b41357fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:38:56.417272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:38:56.418450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:38:56.419621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:38:56.419621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:38:56.419621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:38:56.419621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:38:56.423629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 604/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:38:56.429894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8b78-ae51-4819-9ad4-82a991bd4769 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:38:56.543040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8b78-ae51-4819-9ad4-82a991bd4769 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:38:56.579300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:38:56.592410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac9512-754e-4d24-8a7d-7c17f23e8bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:38:56.593075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac9512-754e-4d24-8a7d-7c17f23e8bdd None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:56.593358 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac9512-754e-4d24-8a7d-7c17f23e8bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:56.593639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac9512-754e-4d24-8a7d-7c17f23e8bdd None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:56.594364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac9512-754e-4d24-8a7d-7c17f23e8bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:38:56.594678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac9512-754e-4d24-8a7d-7c17f23e8bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:38:56.596080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac9512-754e-4d24-8a7d-7c17f23e8bdd None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:38:56.610244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 600/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:38:56.615541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235066a1-419b-4b33-83a4-6c0b41357fac None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:38:56.680781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:38:56.752619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235066a1-419b-4b33-83a4-6c0b41357fac None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:38:56.792826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff51f4-3a5e-4216-aa57-7521cf7697b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:38:56.793541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff51f4-3a5e-4216-aa57-7521cf7697b6 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:56.793824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff51f4-3a5e-4216-aa57-7521cf7697b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:56.794361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff51f4-3a5e-4216-aa57-7521cf7697b6 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:56.795147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff51f4-3a5e-4216-aa57-7521cf7697b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:38:56.796906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff51f4-3a5e-4216-aa57-7521cf7697b6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:38:56.826197 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 605/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Fri Jun 24 23:38:56 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:38:59.145085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c2971-8771-4563-8b6b-f1a0c4d6a61d None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:59.145628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c2971-8771-4563-8b6b-f1a0c4d6a61d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:59.145719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c2971-8771-4563-8b6b-f1a0c4d6a61d None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:59.147292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 601/1206] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:38:59.154525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec1aac-5f51-4fdc-8556-9ed9f8668890 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:59.155003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec1aac-5f51-4fdc-8556-9ed9f8668890 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:59.155083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec1aac-5f51-4fdc-8556-9ed9f8668890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:59.173288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9fec1aac-5f51-4fdc-8556-9ed9f8668890 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:38:59.176513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9fec1aac-5f51-4fdc-8556-9ed9f8668890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:38:59.254738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 606/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:38:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:38:59.280368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c2971-8771-4563-8b6b-f1a0c4d6a61d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:38:59.405164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c2971-8771-4563-8b6b-f1a0c4d6a61d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:38:59.443328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb06616-e904-4a0d-ac87-fb56e5bf34a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:38:59.444194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb06616-e904-4a0d-ac87-fb56e5bf34a7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:59.444477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb06616-e904-4a0d-ac87-fb56e5bf34a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:59.444779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb06616-e904-4a0d-ac87-fb56e5bf34a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:59.457837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb06616-e904-4a0d-ac87-fb56e5bf34a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-fcb06616-e904-4a0d-ac87-fb56e5bf34a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:38:59.467405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:38:59.468625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:38:59.469488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 602/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:38:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:38:59.474908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec1aac-5f51-4fdc-8556-9ed9f8668890 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:38:59.520365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:38:59.589883 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec1aac-5f51-4fdc-8556-9ed9f8668890 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:38:59.627219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3fb50-af95-40f8-b688-4c7d4d757224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:38:59.627917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3fb50-af95-40f8-b688-4c7d4d757224 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:59.628216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3fb50-af95-40f8-b688-4c7d4d757224 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:59.628477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3fb50-af95-40f8-b688-4c7d4d757224 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:59.629249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3fb50-af95-40f8-b688-4c7d4d757224 None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:38:59.629558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3fb50-af95-40f8-b688-4c7d4d757224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:38:59.630932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3fb50-af95-40f8-b688-4c7d4d757224 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:38:59.644627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 607/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:38:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:38:59.646756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb06616-e904-4a0d-ac87-fb56e5bf34a7 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:38:59.707333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:38:59.773878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb06616-e904-4a0d-ac87-fb56e5bf34a7 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:38:59.811163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6c756-d321-4f22-8b21-94639f3ab328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:38:59.812237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6c756-d321-4f22-8b21-94639f3ab328 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:38:59.812516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6c756-d321-4f22-8b21-94639f3ab328 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:38:59.812816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6c756-d321-4f22-8b21-94639f3ab328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:38:59.824779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6c756-d321-4f22-8b21-94639f3ab328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:38:59.826626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6c756-d321-4f22-8b21-94639f3ab328 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-50d6c756-d321-4f22-8b21-94639f3ab328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:38:59.848509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:38:59.849666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:38:59.851172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:38:59.851172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:38:59.851172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:38:59.851172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:38:59.851172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 603/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:38:59.856199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3fb50-af95-40f8-b688-4c7d4d757224 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:38:59.986092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:38:59.989174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3fb50-af95-40f8-b688-4c7d4d757224 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:00.032331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb517746-44ad-49fa-8670-64f366279b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:00.032689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb517746-44ad-49fa-8670-64f366279b58 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:00.032889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb517746-44ad-49fa-8670-64f366279b58 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:00.033045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb517746-44ad-49fa-8670-64f366279b58 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:00.033859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb517746-44ad-49fa-8670-64f366279b58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:00.034202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb517746-44ad-49fa-8670-64f366279b58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:00.035698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb517746-44ad-49fa-8670-64f366279b58 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:00.052486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 608/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:00.059490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6c756-d321-4f22-8b21-94639f3ab328 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:00.185671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6c756-d321-4f22-8b21-94639f3ab328 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:00.187488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:00.242631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5edd12-cc00-4dcf-8ea3-43c1f482baf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:00.243532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5edd12-cc00-4dcf-8ea3-43c1f482baf9 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:00.243818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5edd12-cc00-4dcf-8ea3-43c1f482baf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:00.244151 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5edd12-cc00-4dcf-8ea3-43c1f482baf9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:00.256414 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5edd12-cc00-4dcf-8ea3-43c1f482baf9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:00.258059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5edd12-cc00-4dcf-8ea3-43c1f482baf9 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-7c5edd12-cc00-4dcf-8ea3-43c1f482baf9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:39:00.276977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:00.279674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:00.280657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 604/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:00.286797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb517746-44ad-49fa-8670-64f366279b58 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:00.418156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb517746-44ad-49fa-8670-64f366279b58 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:00.458978 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d57d4-c083-4aa0-88c7-9fc5a647c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:00.459748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d57d4-c083-4aa0-88c7-9fc5a647c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:00.460020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d57d4-c083-4aa0-88c7-9fc5a647c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:00.460378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d57d4-c083-4aa0-88c7-9fc5a647c7f3 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:00.461357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d57d4-c083-4aa0-88c7-9fc5a647c7f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:00.461707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d57d4-c083-4aa0-88c7-9fc5a647c7f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:00.463840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d57d4-c083-4aa0-88c7-9fc5a647c7f3 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:00.492851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 609/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Fri Jun 24 23:39:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:00.494100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5edd12-cc00-4dcf-8ea3-43c1f482baf9 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:00.617767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5edd12-cc00-4dcf-8ea3-43c1f482baf9 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:00.663088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6ec9f-0d98-4d25-91d3-95f961f2bad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:00.663793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ec9f-0d98-4d25-91d3-95f961f2bad4 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:00.664081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ec9f-0d98-4d25-91d3-95f961f2bad4 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:00.664361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6ec9f-0d98-4d25-91d3-95f961f2bad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:00.700774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:00.746932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6ec9f-0d98-4d25-91d3-95f961f2bad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:00.749422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6ec9f-0d98-4d25-91d3-95f961f2bad4 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:00.861188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 605/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:39:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:39:00.867191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d57d4-c083-4aa0-88c7-9fc5a647c7f3 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:00.890929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:00.992276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d57d4-c083-4aa0-88c7-9fc5a647c7f3 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:01.032826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7da12-4a17-4840-8e7a-6773e03a75db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:01.033675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7da12-4a17-4840-8e7a-6773e03a75db None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:01.033953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7da12-4a17-4840-8e7a-6773e03a75db None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:01.034230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7da12-4a17-4840-8e7a-6773e03a75db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:01.045822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7da12-4a17-4840-8e7a-6773e03a75db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:01.047573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7da12-4a17-4840-8e7a-6773e03a75db None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-a5b7da12-4a17-4840-8e7a-6773e03a75db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:01.070222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:01.071228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:01.072102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:01.072102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:01.072102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:01.072102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:01.072102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 610/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:39:01.077883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6ec9f-0d98-4d25-91d3-95f961f2bad4 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:01.147884 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:01.190875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6ec9f-0d98-4d25-91d3-95f961f2bad4 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:01.230546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4f3bf-a4db-4919-8a1f-8d9c9a4dff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:01.231218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4f3bf-a4db-4919-8a1f-8d9c9a4dff26 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:01.231505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4f3bf-a4db-4919-8a1f-8d9c9a4dff26 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:01.231793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4f3bf-a4db-4919-8a1f-8d9c9a4dff26 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:01.232510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4f3bf-a4db-4919-8a1f-8d9c9a4dff26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:01.232854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4f3bf-a4db-4919-8a1f-8d9c9a4dff26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:01.234212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4f3bf-a4db-4919-8a1f-8d9c9a4dff26 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:01.246002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 606/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:01.251609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7da12-4a17-4840-8e7a-6773e03a75db None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:01.343793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:01.384382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7da12-4a17-4840-8e7a-6773e03a75db None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:01.422789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98773d8-d8e5-4ecf-9334-0835d7de325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:01.423518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98773d8-d8e5-4ecf-9334-0835d7de325a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:01.423805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98773d8-d8e5-4ecf-9334-0835d7de325a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:01.424084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98773d8-d8e5-4ecf-9334-0835d7de325a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:01.431122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98773d8-d8e5-4ecf-9334-0835d7de325a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:01.433131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98773d8-d8e5-4ecf-9334-0835d7de325a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-c98773d8-d8e5-4ecf-9334-0835d7de325a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:39:01.447833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:01.449714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:01.450961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:39:01.450961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:39:01.450961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:39:01.450961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:39:01.450961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:39:01.450961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:39:01.450961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:01.450961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:01.450961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 611/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:01.455258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4f3bf-a4db-4919-8a1f-8d9c9a4dff26 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:01.569664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4f3bf-a4db-4919-8a1f-8d9c9a4dff26 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:01.596165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:01.610946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadb9cc-8772-4c0b-8fce-3783cc27fb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:01.611691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadb9cc-8772-4c0b-8fce-3783cc27fb4a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:01.611974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadb9cc-8772-4c0b-8fce-3783cc27fb4a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:01.612346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadb9cc-8772-4c0b-8fce-3783cc27fb4a None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:01.613181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadb9cc-8772-4c0b-8fce-3783cc27fb4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:01.613624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadb9cc-8772-4c0b-8fce-3783cc27fb4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:01.615123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadb9cc-8772-4c0b-8fce-3783cc27fb4a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:01.630410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 607/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:39:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:01.636932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98773d8-d8e5-4ecf-9334-0835d7de325a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:01.764325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98773d8-d8e5-4ecf-9334-0835d7de325a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:01.794850 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:01.803085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c8259-f211-43e9-bf10-fd5462b59cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:01.803755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c8259-f211-43e9-bf10-fd5462b59cc2 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:01.804061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c8259-f211-43e9-bf10-fd5462b59cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:01.804357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c8259-f211-43e9-bf10-fd5462b59cc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:01.805192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c8259-f211-43e9-bf10-fd5462b59cc2 None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=a243d444b7ae487cb0331523d4404d19) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:01.805507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c8259-f211-43e9-bf10-fd5462b59cc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:01.807025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c8259-f211-43e9-bf10-fd5462b59cc2 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:01.812961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 612/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:39:01 2022] GET /identity/v3/role_assignments?user.id=a243d444b7ae487cb0331523d4404d19&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:04.070010 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bb439-7291-4834-85e2-aa753b7d1563 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:04.070695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bb439-7291-4834-85e2-aa753b7d1563 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:04.070695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bb439-7291-4834-85e2-aa753b7d1563 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:04.072193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 608/1220] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:39:04.078765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87493a0-f26c-4b86-8bce-dfb861c9e0e7 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:04.079378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87493a0-f26c-4b86-8bce-dfb861c9e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:04.079460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87493a0-f26c-4b86-8bce-dfb861c9e0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:04.099343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f87493a0-f26c-4b86-8bce-dfb861c9e0e7 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:39:04.100609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f87493a0-f26c-4b86-8bce-dfb861c9e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:39:04.189151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 613/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:39:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:39:04.212153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bb439-7291-4834-85e2-aa753b7d1563 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:04.321879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bb439-7291-4834-85e2-aa753b7d1563 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:04.363308 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9a600-1c46-4c10-a225-47c25a240567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:04.364099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9a600-1c46-4c10-a225-47c25a240567 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:04.364402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9a600-1c46-4c10-a225-47c25a240567 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:04.364692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9a600-1c46-4c10-a225-47c25a240567 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:04.374838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca9a600-1c46-4c10-a225-47c25a240567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-1ca9a600-1c46-4c10-a225-47c25a240567 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:39:04.382469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:04.383467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:04.384322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 609/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:39:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:04.389859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87493a0-f26c-4b86-8bce-dfb861c9e0e7 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:04.491190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87493a0-f26c-4b86-8bce-dfb861c9e0e7 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:04.516330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:04.528659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026539d-0607-485e-ac5f-13fc9d4c74b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:04.529334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026539d-0607-485e-ac5f-13fc9d4c74b6 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:04.529614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026539d-0607-485e-ac5f-13fc9d4c74b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:04.529886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026539d-0607-485e-ac5f-13fc9d4c74b6 None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:04.530627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026539d-0607-485e-ac5f-13fc9d4c74b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:04.530934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026539d-0607-485e-ac5f-13fc9d4c74b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:04.532335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026539d-0607-485e-ac5f-13fc9d4c74b6 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:04.540662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 614/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:39:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:04.547430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9a600-1c46-4c10-a225-47c25a240567 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:04.592570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:04.653256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9a600-1c46-4c10-a225-47c25a240567 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:04.692520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210857b8-bbad-428f-be95-d3b0850aa3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:04.693388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210857b8-bbad-428f-be95-d3b0850aa3cc None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:04.693630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210857b8-bbad-428f-be95-d3b0850aa3cc None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:04.693909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210857b8-bbad-428f-be95-d3b0850aa3cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:04.705576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210857b8-bbad-428f-be95-d3b0850aa3cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:04.707020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210857b8-bbad-428f-be95-d3b0850aa3cc None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-210857b8-bbad-428f-be95-d3b0850aa3cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:04.724573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:04.726079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:04.727198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:04.727198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:04.727198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:04.727198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:04.727198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 610/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:04.732098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026539d-0607-485e-ac5f-13fc9d4c74b6 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:04.840914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026539d-0607-485e-ac5f-13fc9d4c74b6 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:04.875545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5144e-2cd4-457e-b443-49142699da6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:04.876202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5144e-2cd4-457e-b443-49142699da6b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:04.876485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5144e-2cd4-457e-b443-49142699da6b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:04.876809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5144e-2cd4-457e-b443-49142699da6b None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:04.877537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5144e-2cd4-457e-b443-49142699da6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:04.877840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5144e-2cd4-457e-b443-49142699da6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:04.879203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5144e-2cd4-457e-b443-49142699da6b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:04.891618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 615/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:04.897529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210857b8-bbad-428f-be95-d3b0850aa3cc None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:05.006894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:05.014058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210857b8-bbad-428f-be95-d3b0850aa3cc None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:05.052750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a2991-b864-4345-a15b-2ee0ca763243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:05.053555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a2991-b864-4345-a15b-2ee0ca763243 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:05.053838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a2991-b864-4345-a15b-2ee0ca763243 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:05.054117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a2991-b864-4345-a15b-2ee0ca763243 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:05.063079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a2991-b864-4345-a15b-2ee0ca763243 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:05.064669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a2991-b864-4345-a15b-2ee0ca763243 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-495a2991-b864-4345-a15b-2ee0ca763243 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:39:05.080328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:05.081879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:05.082800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 611/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:05.087852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5144e-2cd4-457e-b443-49142699da6b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:05.146532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:05.206985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5144e-2cd4-457e-b443-49142699da6b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:05.245992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259fb0c-7716-47f3-9445-989125d16a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:05.246746 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259fb0c-7716-47f3-9445-989125d16a89 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:05.247048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259fb0c-7716-47f3-9445-989125d16a89 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:05.247392 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259fb0c-7716-47f3-9445-989125d16a89 None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:05.248225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259fb0c-7716-47f3-9445-989125d16a89 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:05.248618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259fb0c-7716-47f3-9445-989125d16a89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:05.250200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259fb0c-7716-47f3-9445-989125d16a89 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:05.270525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 616/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Fri Jun 24 23:39:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:05.276813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a2991-b864-4345-a15b-2ee0ca763243 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:05.396529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a2991-b864-4345-a15b-2ee0ca763243 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:05.432946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c000f6-4db7-47a3-aa11-8e81f6f63f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:05.433786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c000f6-4db7-47a3-aa11-8e81f6f63f2c None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:05.434095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c000f6-4db7-47a3-aa11-8e81f6f63f2c None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:05.434367 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c000f6-4db7-47a3-aa11-8e81f6f63f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:05.445329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c000f6-4db7-47a3-aa11-8e81f6f63f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:05.446923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c000f6-4db7-47a3-aa11-8e81f6f63f2c None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-09c000f6-4db7-47a3-aa11-8e81f6f63f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:05.468130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:05.469338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:05.470695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:05.470695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:05.470695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:05.470695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:05.470695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 612/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:05.475375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259fb0c-7716-47f3-9445-989125d16a89 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:05.587025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259fb0c-7716-47f3-9445-989125d16a89 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:05.589787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:05.625246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da34bdb-c35c-40f3-a22a-cc9c63bda7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:05.625917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da34bdb-c35c-40f3-a22a-cc9c63bda7b0 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:05.626201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da34bdb-c35c-40f3-a22a-cc9c63bda7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:05.626475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da34bdb-c35c-40f3-a22a-cc9c63bda7b0 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:05.627197 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da34bdb-c35c-40f3-a22a-cc9c63bda7b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:05.627506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da34bdb-c35c-40f3-a22a-cc9c63bda7b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:05.629039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da34bdb-c35c-40f3-a22a-cc9c63bda7b0 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:05.641833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 617/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:05.647729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c000f6-4db7-47a3-aa11-8e81f6f63f2c None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:05.660247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:05.766474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c000f6-4db7-47a3-aa11-8e81f6f63f2c None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:05.805339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c66a5-605b-4d53-a1b2-78a6d18f5e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:05.806130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c66a5-605b-4d53-a1b2-78a6d18f5e37 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:05.806408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c66a5-605b-4d53-a1b2-78a6d18f5e37 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:05.807267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c66a5-605b-4d53-a1b2-78a6d18f5e37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:05.814450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c66a5-605b-4d53-a1b2-78a6d18f5e37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:05.815902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c66a5-605b-4d53-a1b2-78a6d18f5e37 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-872c66a5-605b-4d53-a1b2-78a6d18f5e37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:39:05.830261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:05.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:05.832746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:39:05.832746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:39:05.832746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:39:05.832746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:39:05.832746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:39:05.832746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:39:05.832746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:05.832746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:05.832746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 613/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:39:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:05.837373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da34bdb-c35c-40f3-a22a-cc9c63bda7b0 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:05.947037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da34bdb-c35c-40f3-a22a-cc9c63bda7b0 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:05.986216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bfc4a-9bfe-4841-8516-0bff18572e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:05.987100 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bfc4a-9bfe-4841-8516-0bff18572e9b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:05.987447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bfc4a-9bfe-4841-8516-0bff18572e9b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:05.987724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bfc4a-9bfe-4841-8516-0bff18572e9b None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:05.988609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bfc4a-9bfe-4841-8516-0bff18572e9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:05.988987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bfc4a-9bfe-4841-8516-0bff18572e9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:05.990520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bfc4a-9bfe-4841-8516-0bff18572e9b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:06.004262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 618/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:39:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:06.011051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c66a5-605b-4d53-a1b2-78a6d18f5e37 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:06.080115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:06.122788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c66a5-605b-4d53-a1b2-78a6d18f5e37 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:06.158219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb460217-fcee-4568-ae9b-ef450ef4e4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:06.159213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460217-fcee-4568-ae9b-ef450ef4e4af None admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:06.159577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460217-fcee-4568-ae9b-ef450ef4e4af None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:06.159876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460217-fcee-4568-ae9b-ef450ef4e4af None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/a243d444b7ae487cb0331523d4404d19/roles/147f73b08d294eae9859df32606b7d69` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:06.176557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb460217-fcee-4568-ae9b-ef450ef4e4af None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=a243d444b7ae487cb0331523d4404d19, role_id=147f73b08d294eae9859df32606b7d69)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:06.178036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb460217-fcee-4568-ae9b-ef450ef4e4af None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:06.193402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 614/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:39:06 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/a243d444b7ae487cb0331523d4404d19/roles/147f73b08d294eae9859df32606b7d69 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:39:08.239297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8851acc-9e6f-4095-9839-32f44b8e56cb None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:08.239822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8851acc-9e6f-4095-9839-32f44b8e56cb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:08.239822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8851acc-9e6f-4095-9839-32f44b8e56cb None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:08.241178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 619/1233] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:39:08.246743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b01be-9f9f-4bf2-a3c7-869b8e9cef71 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:08.247193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b01be-9f9f-4bf2-a3c7-869b8e9cef71 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:08.247402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b01be-9f9f-4bf2-a3c7-869b8e9cef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:08.265487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-ac3b01be-9f9f-4bf2-a3c7-869b8e9cef71 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:39:08.266888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ac3b01be-9f9f-4bf2-a3c7-869b8e9cef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:39:08.336821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 615/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:39:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 23:39:08.361512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8851acc-9e6f-4095-9839-32f44b8e56cb None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:08.391960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:08.493479 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8851acc-9e6f-4095-9839-32f44b8e56cb None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:08.529751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75032b07-99f9-44b5-9e72-725dccc2b001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:08.530721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75032b07-99f9-44b5-9e72-725dccc2b001 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:08.531096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75032b07-99f9-44b5-9e72-725dccc2b001 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:08.531469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75032b07-99f9-44b5-9e72-725dccc2b001 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:08.542611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75032b07-99f9-44b5-9e72-725dccc2b001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-75032b07-99f9-44b5-9e72-725dccc2b001 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:39:08.550296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:08.551623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:08.552895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 620/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Fri Jun 24 23:39:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:08.558662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b01be-9f9f-4bf2-a3c7-869b8e9cef71 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:08.606067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:08.667116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b01be-9f9f-4bf2-a3c7-869b8e9cef71 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:08.703356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1764d3-9594-4991-a92f-fbfcf365f01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:08.704023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1764d3-9594-4991-a92f-fbfcf365f01a None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:08.704302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1764d3-9594-4991-a92f-fbfcf365f01a None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:08.704612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1764d3-9594-4991-a92f-fbfcf365f01a None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:08.705360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1764d3-9594-4991-a92f-fbfcf365f01a None admin] RBAC: Adding query filter params (name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:08.705666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1764d3-9594-4991-a92f-fbfcf365f01a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:08.707033 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1764d3-9594-4991-a92f-fbfcf365f01a None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:08.716667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 616/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Fri Jun 24 23:39:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:08.722381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75032b07-99f9-44b5-9e72-725dccc2b001 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:08.822874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:08.840044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75032b07-99f9-44b5-9e72-725dccc2b001 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:08.878105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd545ab7-da85-4cea-90e5-58c4939c4a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:08.878972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd545ab7-da85-4cea-90e5-58c4939c4a3d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:08.879261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd545ab7-da85-4cea-90e5-58c4939c4a3d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:08.879578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd545ab7-da85-4cea-90e5-58c4939c4a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:08.893941 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd545ab7-da85-4cea-90e5-58c4939c4a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:08.895420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd545ab7-da85-4cea-90e5-58c4939c4a3d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-cd545ab7-da85-4cea-90e5-58c4939c4a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:08.914043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:08.915150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:08.916846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:08.916846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:08.916846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:08.916846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:08.916846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 621/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:08.921537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1764d3-9594-4991-a92f-fbfcf365f01a None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:09.036602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1764d3-9594-4991-a92f-fbfcf365f01a None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:09.048087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:09.078041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6fb46-5e5d-4d4d-aa70-507ae0bb7f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:09.078642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6fb46-5e5d-4d4d-aa70-507ae0bb7f83 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:09.078931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6fb46-5e5d-4d4d-aa70-507ae0bb7f83 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:09.079309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6fb46-5e5d-4d4d-aa70-507ae0bb7f83 None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:09.080275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6fb46-5e5d-4d4d-aa70-507ae0bb7f83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:09.080701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6fb46-5e5d-4d4d-aa70-507ae0bb7f83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:09.082079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6fb46-5e5d-4d4d-aa70-507ae0bb7f83 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:09.095413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 617/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:09.101770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd545ab7-da85-4cea-90e5-58c4939c4a3d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:09.229464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd545ab7-da85-4cea-90e5-58c4939c4a3d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:09.268673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6e084-b192-41da-b190-fc6c739b804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:09.269512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6e084-b192-41da-b190-fc6c739b804d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:09.269798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6e084-b192-41da-b190-fc6c739b804d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:09.270076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6e084-b192-41da-b190-fc6c739b804d None admin] PATH_INFO: `/v3/users/placement` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:09.273129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:09.279708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6e084-b192-41da-b190-fc6c739b804d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:09.281816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6e084-b192-41da-b190-fc6c739b804d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-fee6e084-b192-41da-b190-fc6c739b804d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:39:09.298932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:09.299903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:09.301032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 622/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:09.307377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6fb46-5e5d-4d4d-aa70-507ae0bb7f83 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:09.420795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6fb46-5e5d-4d4d-aa70-507ae0bb7f83 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:09.458363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf259104-8781-4bcb-b45c-dffcd166eba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:09.459202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf259104-8781-4bcb-b45c-dffcd166eba1 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:09.459551 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf259104-8781-4bcb-b45c-dffcd166eba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:09.459908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf259104-8781-4bcb-b45c-dffcd166eba1 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:09.460845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf259104-8781-4bcb-b45c-dffcd166eba1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:09.461260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf259104-8781-4bcb-b45c-dffcd166eba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:09.463041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf259104-8781-4bcb-b45c-dffcd166eba1 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:09.482965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 618/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Fri Jun 24 23:39:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:09.489036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6e084-b192-41da-b190-fc6c739b804d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:09.604905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6e084-b192-41da-b190-fc6c739b804d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:09.644269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c112db1-cd54-4078-8b8f-f1d88e30b6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:09.645046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c112db1-cd54-4078-8b8f-f1d88e30b6b9 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:09.645414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c112db1-cd54-4078-8b8f-f1d88e30b6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:09.645982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c112db1-cd54-4078-8b8f-f1d88e30b6b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:09.722397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:09.729805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c112db1-cd54-4078-8b8f-f1d88e30b6b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:09.731263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c112db1-cd54-4078-8b8f-f1d88e30b6b9 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:09.832919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 623/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:39:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:39:09.838815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf259104-8781-4bcb-b45c-dffcd166eba1 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:09.868419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:09.950810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf259104-8781-4bcb-b45c-dffcd166eba1 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:09.987676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd8f52-4437-4712-9d80-ff510af71844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:09.988378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8f52-4437-4712-9d80-ff510af71844 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:09.988750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8f52-4437-4712-9d80-ff510af71844 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:09.988976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd8f52-4437-4712-9d80-ff510af71844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:10.000015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd8f52-4437-4712-9d80-ff510af71844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:10.001509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd8f52-4437-4712-9d80-ff510af71844 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-90dd8f52-4437-4712-9d80-ff510af71844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:10.020087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:10.021501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:10.022588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:10.022588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:10.022588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:10.022588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:10.022588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 619/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:10.028384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c112db1-cd54-4078-8b8f-f1d88e30b6b9 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:10.136166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:10.143951 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c112db1-cd54-4078-8b8f-f1d88e30b6b9 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:10.190732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8237d-69d9-4cdc-8312-d1996bfc9d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:10.191510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8237d-69d9-4cdc-8312-d1996bfc9d15 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:10.191982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8237d-69d9-4cdc-8312-d1996bfc9d15 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:10.192319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8237d-69d9-4cdc-8312-d1996bfc9d15 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:10.193444 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8237d-69d9-4cdc-8312-d1996bfc9d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:10.193753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8237d-69d9-4cdc-8312-d1996bfc9d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:10.195943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8237d-69d9-4cdc-8312-d1996bfc9d15 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:10.211580 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 624/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:10.218787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd8f52-4437-4712-9d80-ff510af71844 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:10.309625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:10.332553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd8f52-4437-4712-9d80-ff510af71844 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:10.369208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b16447-a89c-440b-961e-1a4b8011a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:10.370150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b16447-a89c-440b-961e-1a4b8011a6ad None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:10.370443 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b16447-a89c-440b-961e-1a4b8011a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:10.370722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b16447-a89c-440b-961e-1a4b8011a6ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:10.377749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b16447-a89c-440b-961e-1a4b8011a6ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:10.379194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b16447-a89c-440b-961e-1a4b8011a6ad None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-24b16447-a89c-440b-961e-1a4b8011a6ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:39:10.393725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:10.394999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:10.396339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:39:10.396339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:39:10.396339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:39:10.396339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:39:10.396339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:39:10.396339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:39:10.396339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:10.396339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:10.396339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 620/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:39:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:10.401369 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8237d-69d9-4cdc-8312-d1996bfc9d15 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:10.517460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8237d-69d9-4cdc-8312-d1996bfc9d15 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:10.551376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825cfd1-038d-402e-9fb6-fb2902337261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:10.552207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825cfd1-038d-402e-9fb6-fb2902337261 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:10.552655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825cfd1-038d-402e-9fb6-fb2902337261 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:10.553095 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825cfd1-038d-402e-9fb6-fb2902337261 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:10.554117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825cfd1-038d-402e-9fb6-fb2902337261 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:10.554560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825cfd1-038d-402e-9fb6-fb2902337261 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:10.556359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825cfd1-038d-402e-9fb6-fb2902337261 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:10.570106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 625/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:39:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:39:10.577475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b16447-a89c-440b-961e-1a4b8011a6ad None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:10.687850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b16447-a89c-440b-961e-1a4b8011a6ad None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:10.725791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a0052-fd2f-4dee-a142-69863d1579dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:10.726591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a0052-fd2f-4dee-a142-69863d1579dc None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:10.726948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a0052-fd2f-4dee-a142-69863d1579dc None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:10.727286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a0052-fd2f-4dee-a142-69863d1579dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:10.728202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a0052-fd2f-4dee-a142-69863d1579dc None admin] RBAC: Adding query filter params (role.id=147f73b08d294eae9859df32606b7d69, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=a243d444b7ae487cb0331523d4404d19) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:10.728594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a0052-fd2f-4dee-a142-69863d1579dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:10.730403 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a0052-fd2f-4dee-a142-69863d1579dc None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:10.737139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 621/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:39:10 2022] GET /identity/v3/role_assignments?user.id=a243d444b7ae487cb0331523d4404d19&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=147f73b08d294eae9859df32606b7d69 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:12.850457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5a690-79da-402b-bb04-c47255c60959 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:12.851035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5a690-79da-402b-bb04-c47255c60959 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:12.851035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5a690-79da-402b-bb04-c47255c60959 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:12.852382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 626/1247] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:39:12.858258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476bae-0b06-4b2a-a973-d67b9803097a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:12.858680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476bae-0b06-4b2a-a973-d67b9803097a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:12.858806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476bae-0b06-4b2a-a973-d67b9803097a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:12.864866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:12.879858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-e6476bae-0b06-4b2a-a973-d67b9803097a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:39:12.881278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e6476bae-0b06-4b2a-a973-d67b9803097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:39:12.968609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 622/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:39:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:39:12.987014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5a690-79da-402b-bb04-c47255c60959 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:13.007915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:13.120856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5a690-79da-402b-bb04-c47255c60959 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:13.163892 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef814823-811e-46c5-b7a4-79b77cdc9382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:13.164734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef814823-811e-46c5-b7a4-79b77cdc9382 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:13.165020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef814823-811e-46c5-b7a4-79b77cdc9382 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:13.165299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef814823-811e-46c5-b7a4-79b77cdc9382 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:13.177809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef814823-811e-46c5-b7a4-79b77cdc9382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-ef814823-811e-46c5-b7a4-79b77cdc9382 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:39:13.189641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:13.190808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:39:13.191847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:13.195836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 627/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:39:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:13.197928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6476bae-0b06-4b2a-a973-d67b9803097a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:13.317987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6476bae-0b06-4b2a-a973-d67b9803097a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:13.357038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:13.364056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c35ff-d29c-495f-bc27-c30bfbeb5c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:13.364765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c35ff-d29c-495f-bc27-c30bfbeb5c66 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:13.365050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c35ff-d29c-495f-bc27-c30bfbeb5c66 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:13.365333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c35ff-d29c-495f-bc27-c30bfbeb5c66 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:13.366258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c35ff-d29c-495f-bc27-c30bfbeb5c66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:13.366748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c35ff-d29c-495f-bc27-c30bfbeb5c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:13.368610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c35ff-d29c-495f-bc27-c30bfbeb5c66 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:13.378616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 623/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:13.385306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef814823-811e-46c5-b7a4-79b77cdc9382 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:13.475979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:13.503074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef814823-811e-46c5-b7a4-79b77cdc9382 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:13.542047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875765b7-0aab-4eb3-9c30-171ae168b96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:13.542966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875765b7-0aab-4eb3-9c30-171ae168b96d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:13.543292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875765b7-0aab-4eb3-9c30-171ae168b96d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:13.543632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875765b7-0aab-4eb3-9c30-171ae168b96d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:13.557271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875765b7-0aab-4eb3-9c30-171ae168b96d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:13.559028 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875765b7-0aab-4eb3-9c30-171ae168b96d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-875765b7-0aab-4eb3-9c30-171ae168b96d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:13.578843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:13.579952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:13.580903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:13.580903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:13.580903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:13.580903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:13.580903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 628/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:39:13.586122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c35ff-d29c-495f-bc27-c30bfbeb5c66 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:13.720696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c35ff-d29c-495f-bc27-c30bfbeb5c66 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:13.760682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd9952-7fc6-4f0a-a5f2-c9617c6dc24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:13.761363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd9952-7fc6-4f0a-a5f2-c9617c6dc24b None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:13.761652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd9952-7fc6-4f0a-a5f2-c9617c6dc24b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:13.762003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd9952-7fc6-4f0a-a5f2-c9617c6dc24b None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:13.762806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd9952-7fc6-4f0a-a5f2-c9617c6dc24b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:13.763133 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd9952-7fc6-4f0a-a5f2-c9617c6dc24b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:13.764652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd9952-7fc6-4f0a-a5f2-c9617c6dc24b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:13.778561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 624/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:13.785503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875765b7-0aab-4eb3-9c30-171ae168b96d None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:13.896375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875765b7-0aab-4eb3-9c30-171ae168b96d None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:13.925608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:13.934969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e2879-bc45-4471-85d0-a2292d575a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:13.935809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e2879-bc45-4471-85d0-a2292d575a06 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:13.936092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e2879-bc45-4471-85d0-a2292d575a06 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:13.936370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e2879-bc45-4471-85d0-a2292d575a06 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:13.945753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e2879-bc45-4471-85d0-a2292d575a06 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:13.947814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e2879-bc45-4471-85d0-a2292d575a06 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-995e2879-bc45-4471-85d0-a2292d575a06 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:39:13.963605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:13.964770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:13.965803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 629/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:13.970684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd9952-7fc6-4f0a-a5f2-c9617c6dc24b None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:14.035901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:14.085767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd9952-7fc6-4f0a-a5f2-c9617c6dc24b None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:14.130459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293ce0c-7ead-4128-8431-963be42daa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:14.131136 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293ce0c-7ead-4128-8431-963be42daa82 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:14.131423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293ce0c-7ead-4128-8431-963be42daa82 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:14.131706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293ce0c-7ead-4128-8431-963be42daa82 None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:14.133402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293ce0c-7ead-4128-8431-963be42daa82 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:14.133826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293ce0c-7ead-4128-8431-963be42daa82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:14.136299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293ce0c-7ead-4128-8431-963be42daa82 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:14.161746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 625/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Fri Jun 24 23:39:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:14.163350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e2879-bc45-4471-85d0-a2292d575a06 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:14.277011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e2879-bc45-4471-85d0-a2292d575a06 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:14.317048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351144bb-bb78-419a-bff7-34a22f180741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:14.317937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351144bb-bb78-419a-bff7-34a22f180741 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:14.318398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351144bb-bb78-419a-bff7-34a22f180741 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:14.318676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351144bb-bb78-419a-bff7-34a22f180741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:14.383867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:14.398332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351144bb-bb78-419a-bff7-34a22f180741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:14.400688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351144bb-bb78-419a-bff7-34a22f180741 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:14.513365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 630/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:39:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 23:39:14.519486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293ce0c-7ead-4128-8431-963be42daa82 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:14.636483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293ce0c-7ead-4128-8431-963be42daa82 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:14.653703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:14.680624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cdaab-50b8-4fbe-a51c-a472aa12dc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:14.682022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cdaab-50b8-4fbe-a51c-a472aa12dc56 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:14.682371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cdaab-50b8-4fbe-a51c-a472aa12dc56 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:14.682929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cdaab-50b8-4fbe-a51c-a472aa12dc56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:14.695525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cdaab-50b8-4fbe-a51c-a472aa12dc56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:14.698230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cdaab-50b8-4fbe-a51c-a472aa12dc56 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-676cdaab-50b8-4fbe-a51c-a472aa12dc56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:14.719452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:14.720560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:14.721683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:14.721683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:14.721683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:14.721683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:14.726184 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 626/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:14.727535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351144bb-bb78-419a-bff7-34a22f180741 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:14.828854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:14.840962 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351144bb-bb78-419a-bff7-34a22f180741 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:14.878199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d0dce-8321-4c7f-8f1b-27239d119c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:14.879050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d0dce-8321-4c7f-8f1b-27239d119c25 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:14.879350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d0dce-8321-4c7f-8f1b-27239d119c25 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:14.879722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d0dce-8321-4c7f-8f1b-27239d119c25 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:14.880606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d0dce-8321-4c7f-8f1b-27239d119c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:14.880949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d0dce-8321-4c7f-8f1b-27239d119c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:14.882685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d0dce-8321-4c7f-8f1b-27239d119c25 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:14.896462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 631/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:14.902826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cdaab-50b8-4fbe-a51c-a472aa12dc56 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:15.017716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cdaab-50b8-4fbe-a51c-a472aa12dc56 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:15.053436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baa6c5-1157-4910-b1ac-3b8defcaced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:15.054291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa6c5-1157-4910-b1ac-3b8defcaced3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:15.054577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa6c5-1157-4910-b1ac-3b8defcaced3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:15.054871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96baa6c5-1157-4910-b1ac-3b8defcaced3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:15.062042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96baa6c5-1157-4910-b1ac-3b8defcaced3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:15.063494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96baa6c5-1157-4910-b1ac-3b8defcaced3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-96baa6c5-1157-4910-b1ac-3b8defcaced3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:39:15.079630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:15.080730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:15.081835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:39:15.081835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:39:15.081835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:39:15.081835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:39:15.081835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:39:15.081835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:39:15.081835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:15.081835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:15.081835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 627/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:39:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:15.087204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d0dce-8321-4c7f-8f1b-27239d119c25 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:15.201556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d0dce-8321-4c7f-8f1b-27239d119c25 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:15.237661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5028af-3aef-484a-a58f-fcc961daeff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:15.238330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5028af-3aef-484a-a58f-fcc961daeff8 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:15.238609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5028af-3aef-484a-a58f-fcc961daeff8 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:15.238881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5028af-3aef-484a-a58f-fcc961daeff8 None admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:15.239661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5028af-3aef-484a-a58f-fcc961daeff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:15.239974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5028af-3aef-484a-a58f-fcc961daeff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:15.241461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5028af-3aef-484a-a58f-fcc961daeff8 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:15.258000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 632/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:39:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:15.265868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baa6c5-1157-4910-b1ac-3b8defcaced3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:15.293614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:15.375742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96baa6c5-1157-4910-b1ac-3b8defcaced3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:15.414430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62b8ff-d566-4c66-b7f0-791939237f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:15.415136 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62b8ff-d566-4c66-b7f0-791939237f79 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:15.415438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62b8ff-d566-4c66-b7f0-791939237f79 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:15.415761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62b8ff-d566-4c66-b7f0-791939237f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:15.416748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62b8ff-d566-4c66-b7f0-791939237f79 None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=a243d444b7ae487cb0331523d4404d19) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:15.417068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62b8ff-d566-4c66-b7f0-791939237f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:15.418517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62b8ff-d566-4c66-b7f0-791939237f79 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:15.426015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 628/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:39:15 2022] GET /identity/v3/role_assignments?user.id=a243d444b7ae487cb0331523d4404d19&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:17.533040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8fef2-981b-4a77-b899-7d29bf0cc97f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:17.533591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8fef2-981b-4a77-b899-7d29bf0cc97f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:17.533591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8fef2-981b-4a77-b899-7d29bf0cc97f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:17.535004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 633/1261] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:39:17.540045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b87ed-2721-43e9-abfa-e54cb312a60a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:17.540474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b87ed-2721-43e9-abfa-e54cb312a60a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:17.541184 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b87ed-2721-43e9-abfa-e54cb312a60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:17.558500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-742b87ed-2721-43e9-abfa-e54cb312a60a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:39:17.559677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-742b87ed-2721-43e9-abfa-e54cb312a60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:39:17.625470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 629/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:39:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:39:17.648671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8fef2-981b-4a77-b899-7d29bf0cc97f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:17.661945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:17.760111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8fef2-981b-4a77-b899-7d29bf0cc97f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:17.793738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e2872-3b0d-4fd3-8566-556431bd1b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:17.794456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e2872-3b0d-4fd3-8566-556431bd1b25 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:17.794732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e2872-3b0d-4fd3-8566-556431bd1b25 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:17.795010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e2872-3b0d-4fd3-8566-556431bd1b25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:17.804653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e2872-3b0d-4fd3-8566-556431bd1b25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-f06e2872-3b0d-4fd3-8566-556431bd1b25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:39:17.812246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:17.813350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:17.814373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 634/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 23:39:17.819685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b87ed-2721-43e9-abfa-e54cb312a60a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:17.838148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:17.915663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b87ed-2721-43e9-abfa-e54cb312a60a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:17.949745 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603633f-15e6-4782-a22d-5f5b0d102746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:17.950413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603633f-15e6-4782-a22d-5f5b0d102746 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:17.950696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603633f-15e6-4782-a22d-5f5b0d102746 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:17.950971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603633f-15e6-4782-a22d-5f5b0d102746 None admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:17.951717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603633f-15e6-4782-a22d-5f5b0d102746 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:17.952024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603633f-15e6-4782-a22d-5f5b0d102746 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:17.953415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603633f-15e6-4782-a22d-5f5b0d102746 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:17.961393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 630/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:17.967477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e2872-3b0d-4fd3-8566-556431bd1b25 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:18.032477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:18.064378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e2872-3b0d-4fd3-8566-556431bd1b25 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:18.096847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dc863-12a5-4c89-89e9-7c0779882480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:18.097622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dc863-12a5-4c89-89e9-7c0779882480 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:18.097904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dc863-12a5-4c89-89e9-7c0779882480 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:18.098174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dc863-12a5-4c89-89e9-7c0779882480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:18.109161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dc863-12a5-4c89-89e9-7c0779882480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:18.110587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dc863-12a5-4c89-89e9-7c0779882480 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-859dc863-12a5-4c89-89e9-7c0779882480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:18.127702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:18.128741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:18.129821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:18.129821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:18.129821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:18.129821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:18.129821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 635/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:18.134657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603633f-15e6-4782-a22d-5f5b0d102746 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:18.214596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:18.233900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603633f-15e6-4782-a22d-5f5b0d102746 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:18.265897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81784615-d94f-4f00-a7a5-430e88aa0fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:18.266552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81784615-d94f-4f00-a7a5-430e88aa0fde None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:18.266837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81784615-d94f-4f00-a7a5-430e88aa0fde None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:18.267116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81784615-d94f-4f00-a7a5-430e88aa0fde None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:18.267834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81784615-d94f-4f00-a7a5-430e88aa0fde None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:18.268146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81784615-d94f-4f00-a7a5-430e88aa0fde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:18.269538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81784615-d94f-4f00-a7a5-430e88aa0fde None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:18.282381 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 631/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:18.287925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dc863-12a5-4c89-89e9-7c0779882480 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:18.383558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dc863-12a5-4c89-89e9-7c0779882480 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:18.402908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:18.429176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293d4cb-ea92-42df-a579-9b1bb6b6720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:18.429977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293d4cb-ea92-42df-a579-9b1bb6b6720a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:18.430259 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293d4cb-ea92-42df-a579-9b1bb6b6720a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:18.430534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293d4cb-ea92-42df-a579-9b1bb6b6720a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:18.443256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293d4cb-ea92-42df-a579-9b1bb6b6720a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:18.444721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293d4cb-ea92-42df-a579-9b1bb6b6720a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-2293d4cb-ea92-42df-a579-9b1bb6b6720a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:39:18.458691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:18.459867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:18.460989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 636/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:18.466129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81784615-d94f-4f00-a7a5-430e88aa0fde None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:18.564747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81784615-d94f-4f00-a7a5-430e88aa0fde None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:18.597907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbbe67-446a-4d14-a25c-7f8765ed98fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:18.598567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbbe67-446a-4d14-a25c-7f8765ed98fe None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:18.598848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbbe67-446a-4d14-a25c-7f8765ed98fe None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:18.599123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbbe67-446a-4d14-a25c-7f8765ed98fe None admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:18.599955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbbe67-446a-4d14-a25c-7f8765ed98fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:18.600264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbbe67-446a-4d14-a25c-7f8765ed98fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:18.601705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbbe67-446a-4d14-a25c-7f8765ed98fe None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:18.610536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:18.619139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 632/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Fri Jun 24 23:39:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:18.624677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293d4cb-ea92-42df-a579-9b1bb6b6720a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:18.734012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293d4cb-ea92-42df-a579-9b1bb6b6720a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:18.766475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe30a7-74ac-4d3f-839e-412065afdad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:18.767460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe30a7-74ac-4d3f-839e-412065afdad9 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:18.767745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe30a7-74ac-4d3f-839e-412065afdad9 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:18.768026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe30a7-74ac-4d3f-839e-412065afdad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:18.781797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe30a7-74ac-4d3f-839e-412065afdad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:18.783880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe30a7-74ac-4d3f-839e-412065afdad9 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-ebfe30a7-74ac-4d3f-839e-412065afdad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:18.801499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:18.802584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:18.803683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:18.803683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:18.803683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:18.803683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:18.803683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 637/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:18.808702 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbbe67-446a-4d14-a25c-7f8765ed98fe None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:18.906892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbbe67-446a-4d14-a25c-7f8765ed98fe None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:18.938968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e8782-0c22-4100-9cf6-26a716e5bddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:18.939625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e8782-0c22-4100-9cf6-26a716e5bddb None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:18.939908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e8782-0c22-4100-9cf6-26a716e5bddb None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:18.940184 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e8782-0c22-4100-9cf6-26a716e5bddb None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:18.940941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e8782-0c22-4100-9cf6-26a716e5bddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:18.941249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e8782-0c22-4100-9cf6-26a716e5bddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:18.942607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e8782-0c22-4100-9cf6-26a716e5bddb None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:18.954247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 633/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:18.960199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe30a7-74ac-4d3f-839e-412065afdad9 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:18.991277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:19.058587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe30a7-74ac-4d3f-839e-412065afdad9 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:19.090756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2bf8a-4d07-4549-9302-dd1e22a1c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:19.091530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2bf8a-4d07-4549-9302-dd1e22a1c36a None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:19.091813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2bf8a-4d07-4549-9302-dd1e22a1c36a None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:19.092091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2bf8a-4d07-4549-9302-dd1e22a1c36a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:19.098517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2bf8a-4d07-4549-9302-dd1e22a1c36a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:19.099949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2bf8a-4d07-4549-9302-dd1e22a1c36a None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-fcc2bf8a-4d07-4549-9302-dd1e22a1c36a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:39:19.113856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:19.114923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:19.116014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:39:19.116014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:39:19.116014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:39:19.116014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:39:19.116014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:39:19.116014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:39:19.116014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:19.116014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:19.116014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 638/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:39:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:19.121304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e8782-0c22-4100-9cf6-26a716e5bddb None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:19.190222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:19.231081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e8782-0c22-4100-9cf6-26a716e5bddb None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:19.263928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fed5d-2b57-445e-9140-23175a7b8bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:19.264526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fed5d-2b57-445e-9140-23175a7b8bc3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:19.264834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fed5d-2b57-445e-9140-23175a7b8bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:19.265106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fed5d-2b57-445e-9140-23175a7b8bc3 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:19.265877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fed5d-2b57-445e-9140-23175a7b8bc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:19.266182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fed5d-2b57-445e-9140-23175a7b8bc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:19.267584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96fed5d-2b57-445e-9140-23175a7b8bc3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:19.279235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 634/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:39:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:19.285195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2bf8a-4d07-4549-9302-dd1e22a1c36a None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:19.375340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:19.384105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2bf8a-4d07-4549-9302-dd1e22a1c36a None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:19.422498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8db0a8-f0a5-4c48-a9f9-c860b3223135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:19.422498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8db0a8-f0a5-4c48-a9f9-c860b3223135 None admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:19.423524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8db0a8-f0a5-4c48-a9f9-c860b3223135 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:19.423810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8db0a8-f0a5-4c48-a9f9-c860b3223135 None admin] PATH_INFO: `/v3/projects/c9a1b392b7034790a51237458cab8b75/users/a243d444b7ae487cb0331523d4404d19/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:19.425803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8db0a8-f0a5-4c48-a9f9-c860b3223135 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a1b392b7034790a51237458cab8b75, user_id=a243d444b7ae487cb0331523d4404d19, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:19.427230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8db0a8-f0a5-4c48-a9f9-c860b3223135 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:19.440343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 639/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Fri Jun 24 23:39:19 2022] PUT /identity/v3/projects/c9a1b392b7034790a51237458cab8b75/users/a243d444b7ae487cb0331523d4404d19/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:39:21.440861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09341dc-f761-485a-a132-83ebef01f9c1 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:21.441453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09341dc-f761-485a-a132-83ebef01f9c1 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:21.441453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09341dc-f761-485a-a132-83ebef01f9c1 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:21.442773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 635/1274] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:39:21.448364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d857b-c4f5-4562-b0db-2242d14aead9 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:21.448784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d857b-c4f5-4562-b0db-2242d14aead9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:21.448954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d857b-c4f5-4562-b0db-2242d14aead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:21.465331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-091d857b-c4f5-4562-b0db-2242d14aead9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:39:21.466524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-091d857b-c4f5-4562-b0db-2242d14aead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:39:21.531030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 640/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:39:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:39:21.553945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09341dc-f761-485a-a132-83ebef01f9c1 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:21.656298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09341dc-f761-485a-a132-83ebef01f9c1 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:21.676070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:21.693526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fef8e-5fda-4652-b101-3b752af09527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:21.694251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fef8e-5fda-4652-b101-3b752af09527 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:21.694532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fef8e-5fda-4652-b101-3b752af09527 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:21.694806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fef8e-5fda-4652-b101-3b752af09527 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:21.705034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fef8e-5fda-4652-b101-3b752af09527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-ec0fef8e-5fda-4652-b101-3b752af09527 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 23:39:21.712735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:21.713919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:21.715002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 636/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Fri Jun 24 23:39:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:21.719865 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d857b-c4f5-4562-b0db-2242d14aead9 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:21.798754 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:21.819959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d857b-c4f5-4562-b0db-2242d14aead9 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:21.857046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3ef8c-56f9-4e4e-9f8b-fb9d6a8fe21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:21.857712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ef8c-56f9-4e4e-9f8b-fb9d6a8fe21b None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:21.857999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ef8c-56f9-4e4e-9f8b-fb9d6a8fe21b None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:21.858280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ef8c-56f9-4e4e-9f8b-fb9d6a8fe21b None admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:21.859031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3ef8c-56f9-4e4e-9f8b-fb9d6a8fe21b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:21.859344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3ef8c-56f9-4e4e-9f8b-fb9d6a8fe21b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:21.860747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3ef8c-56f9-4e4e-9f8b-fb9d6a8fe21b None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:21.869426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 641/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Fri Jun 24 23:39:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:21.875218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fef8e-5fda-4652-b101-3b752af09527 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:21.974969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fef8e-5fda-4652-b101-3b752af09527 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:22.008108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215f110-8a77-4002-82ec-d00223e298bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:22.008960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215f110-8a77-4002-82ec-d00223e298bf None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:22.009247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215f110-8a77-4002-82ec-d00223e298bf None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:22.009525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215f110-8a77-4002-82ec-d00223e298bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:22.019614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215f110-8a77-4002-82ec-d00223e298bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:22.021329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215f110-8a77-4002-82ec-d00223e298bf None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-2215f110-8a77-4002-82ec-d00223e298bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:22.038393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:22.039423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:22.041246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:22.041246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:22.041246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:22.041246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:22.041246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 637/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:22.046125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3ef8c-56f9-4e4e-9f8b-fb9d6a8fe21b None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:22.153637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3ef8c-56f9-4e4e-9f8b-fb9d6a8fe21b None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:22.187074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632101e-fdce-40c5-b630-45dffeb34d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:22.187736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632101e-fdce-40c5-b630-45dffeb34d20 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:22.188020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632101e-fdce-40c5-b630-45dffeb34d20 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:22.188296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632101e-fdce-40c5-b630-45dffeb34d20 None admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:22.189047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632101e-fdce-40c5-b630-45dffeb34d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:22.189362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632101e-fdce-40c5-b630-45dffeb34d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:22.190737 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632101e-fdce-40c5-b630-45dffeb34d20 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:22.202434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 642/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:22.208206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215f110-8a77-4002-82ec-d00223e298bf None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:22.239423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:22.306523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215f110-8a77-4002-82ec-d00223e298bf None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:22.339028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362972e-d9a7-4471-822b-84bfd8b2ed6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:22.339811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362972e-d9a7-4471-822b-84bfd8b2ed6f None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:22.340090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362972e-d9a7-4471-822b-84bfd8b2ed6f None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:22.340361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362972e-d9a7-4471-822b-84bfd8b2ed6f None admin] PATH_INFO: `/v3/users/placement` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:22.347559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362972e-d9a7-4471-822b-84bfd8b2ed6f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:22.349027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362972e-d9a7-4471-822b-84bfd8b2ed6f None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-c362972e-d9a7-4471-822b-84bfd8b2ed6f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 23:39:22.362999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:22.364085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:22.365309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 638/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:22.370041 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632101e-fdce-40c5-b630-45dffeb34d20 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:22.373763 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:22.475832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632101e-fdce-40c5-b630-45dffeb34d20 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:22.509864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8522-916e-4877-9961-bdd0d592e6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:22.510454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8522-916e-4877-9961-bdd0d592e6bd None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:22.510738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8522-916e-4877-9961-bdd0d592e6bd None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:22.511016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e8522-916e-4877-9961-bdd0d592e6bd None admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:22.511854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e8522-916e-4877-9961-bdd0d592e6bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:22.512166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e8522-916e-4877-9961-bdd0d592e6bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:22.513623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e8522-916e-4877-9961-bdd0d592e6bd None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:22.531062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 643/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Fri Jun 24 23:39:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:22.536890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362972e-d9a7-4471-822b-84bfd8b2ed6f None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:22.623049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:22.632261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362972e-d9a7-4471-822b-84bfd8b2ed6f None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:22.673985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f25f9e-0e5d-46f7-af4b-3645728adad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:22.674627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f25f9e-0e5d-46f7-af4b-3645728adad3 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:22.674909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f25f9e-0e5d-46f7-af4b-3645728adad3 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:22.675184 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f25f9e-0e5d-46f7-af4b-3645728adad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:22.739400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f25f9e-0e5d-46f7-af4b-3645728adad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:22.740868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f25f9e-0e5d-46f7-af4b-3645728adad3 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:22.828701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:22.839354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 639/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Fri Jun 24 23:39:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 23:39:22.845108 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8522-916e-4877-9961-bdd0d592e6bd None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:22.906330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:22.943596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e8522-916e-4877-9961-bdd0d592e6bd None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:22.976890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93d58-aba5-44d9-99c7-00a618b170f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:22.977728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93d58-aba5-44d9-99c7-00a618b170f3 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:22.978032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93d58-aba5-44d9-99c7-00a618b170f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:22.978333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b93d58-aba5-44d9-99c7-00a618b170f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:22.987980 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b93d58-aba5-44d9-99c7-00a618b170f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:22.989584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b93d58-aba5-44d9-99c7-00a618b170f3 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-33b93d58-aba5-44d9-99c7-00a618b170f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 23:39:23.007591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:23.008619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:23.010297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 23:39:23.010297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 23:39:23.010297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 23:39:23.010297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:23.012753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 644/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:23.015068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f25f9e-0e5d-46f7-af4b-3645728adad3 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:23.112281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f25f9e-0e5d-46f7-af4b-3645728adad3 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:23.145650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96380d3f-ff68-4e5b-b204-a0b5c0cbec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:23.146308 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96380d3f-ff68-4e5b-b204-a0b5c0cbec0e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:23.146592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96380d3f-ff68-4e5b-b204-a0b5c0cbec0e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:23.146870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96380d3f-ff68-4e5b-b204-a0b5c0cbec0e None admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:23.147582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96380d3f-ff68-4e5b-b204-a0b5c0cbec0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:23.147892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96380d3f-ff68-4e5b-b204-a0b5c0cbec0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:23.149274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96380d3f-ff68-4e5b-b204-a0b5c0cbec0e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:23.160161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 640/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Fri Jun 24 23:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:23.166613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93d58-aba5-44d9-99c7-00a618b170f3 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:23.270251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b93d58-aba5-44d9-99c7-00a618b170f3 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:23.284204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:23.304729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453528e-a5ea-45d5-ac21-d6b352aa260c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:23.305511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453528e-a5ea-45d5-ac21-d6b352aa260c None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:23.306924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453528e-a5ea-45d5-ac21-d6b352aa260c None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:23.307210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453528e-a5ea-45d5-ac21-d6b352aa260c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:23.313648 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453528e-a5ea-45d5-ac21-d6b352aa260c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:23.315090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453528e-a5ea-45d5-ac21-d6b352aa260c None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application [None req-9453528e-a5ea-45d5-ac21-d6b352aa260c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 23:39:23.328919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:23.329965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:23.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 23:39:23.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 23:39:23.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 23:39:23.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 23:39:23.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 23:39:23.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 23:39:23.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 23:39:23.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: ERROR keystone.server.flask.application Jun 24 23:39:23.331215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 645/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Fri Jun 24 23:39:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:23.336034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96380d3f-ff68-4e5b-b204-a0b5c0cbec0e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:23.385944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:23.434911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96380d3f-ff68-4e5b-b204-a0b5c0cbec0e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:23.467130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4b734-9290-4321-8c61-4cce312926b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:23.467777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4b734-9290-4321-8c61-4cce312926b0 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:23.468062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4b734-9290-4321-8c61-4cce312926b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:23.468341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4b734-9290-4321-8c61-4cce312926b0 None admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:23.469159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4b734-9290-4321-8c61-4cce312926b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:23.469469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4b734-9290-4321-8c61-4cce312926b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:23.470868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f4b734-9290-4321-8c61-4cce312926b0 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:23.483339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 641/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Fri Jun 24 23:39:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:23.489346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453528e-a5ea-45d5-ac21-d6b352aa260c None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:23.588502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453528e-a5ea-45d5-ac21-d6b352aa260c None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:23.620973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18c499-eb8c-45f0-9668-eff47b1b1950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:23.621623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18c499-eb8c-45f0-9668-eff47b1b1950 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:23.621912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18c499-eb8c-45f0-9668-eff47b1b1950 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:23.622196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18c499-eb8c-45f0-9668-eff47b1b1950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:23.623037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18c499-eb8c-45f0-9668-eff47b1b1950 None admin] RBAC: Adding query filter params (role.id=419ec418f6544c938d3669a80f7b2ffd, scope.project.id=c9a1b392b7034790a51237458cab8b75, user.id=a243d444b7ae487cb0331523d4404d19) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:23.623354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18c499-eb8c-45f0-9668-eff47b1b1950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:23.624808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18c499-eb8c-45f0-9668-eff47b1b1950 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:23.630590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 646/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Fri Jun 24 23:39:23 2022] GET /identity/v3/role_assignments?user.id=a243d444b7ae487cb0331523d4404d19&scope.project.id=c9a1b392b7034790a51237458cab8b75&role.id=419ec418f6544c938d3669a80f7b2ffd => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:25.655701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3dadb-c725-4b35-901d-0d01d6c85b8e None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:25.656422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3dadb-c725-4b35-901d-0d01d6c85b8e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:25.656422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3dadb-c725-4b35-901d-0d01d6c85b8e None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:25.657623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 642/1288] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:39:25.663599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110297af-ec6f-4f13-b6ac-e7815d2fd863 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:25.664075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110297af-ec6f-4f13-b6ac-e7815d2fd863 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:25.664286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110297af-ec6f-4f13-b6ac-e7815d2fd863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:25.684953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-110297af-ec6f-4f13-b6ac-e7815d2fd863 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:39:25.686673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-110297af-ec6f-4f13-b6ac-e7815d2fd863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:39:25.721596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:25.757692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 647/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:39:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:39:25.780130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3dadb-c725-4b35-901d-0d01d6c85b8e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:25.877734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3dadb-c725-4b35-901d-0d01d6c85b8e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:25.887930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:25.911994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea4ae6-03f7-4316-a4e1-127a72226a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:25.912751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea4ae6-03f7-4316-a4e1-127a72226a23 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:25.913040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea4ae6-03f7-4316-a4e1-127a72226a23 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:25.913320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea4ae6-03f7-4316-a4e1-127a72226a23 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:25.913976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea4ae6-03f7-4316-a4e1-127a72226a23 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:25.915958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea4ae6-03f7-4316-a4e1-127a72226a23 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-feea4ae6-03f7-4316-a4e1-127a72226a23 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:25.927547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:25.928793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:25.929837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 643/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:39:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:25.934693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110297af-ec6f-4f13-b6ac-e7815d2fd863 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:26.030109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110297af-ec6f-4f13-b6ac-e7815d2fd863 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:26.062485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec57e69-4eb9-4b05-a3be-acfd21b75d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:26.063131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec57e69-4eb9-4b05-a3be-acfd21b75d0d None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:26.063411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec57e69-4eb9-4b05-a3be-acfd21b75d0d None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:26.063684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec57e69-4eb9-4b05-a3be-acfd21b75d0d None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:26.064380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec57e69-4eb9-4b05-a3be-acfd21b75d0d None admin] RBAC: Adding query filter params (name=placement) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:26.064716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec57e69-4eb9-4b05-a3be-acfd21b75d0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:26.066077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec57e69-4eb9-4b05-a3be-acfd21b75d0d None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:26.070743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 648/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Fri Jun 24 23:39:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:26.076463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea4ae6-03f7-4316-a4e1-127a72226a23 None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:26.192754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea4ae6-03f7-4316-a4e1-127a72226a23 None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:26.226035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d72c19-533c-4367-acb7-07eb2bcaacc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:26.226691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d72c19-533c-4367-acb7-07eb2bcaacc7 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:26.226974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d72c19-533c-4367-acb7-07eb2bcaacc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:26.227250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d72c19-533c-4367-acb7-07eb2bcaacc7 None admin] PATH_INFO: `/v3/services` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:26.228326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d72c19-533c-4367-acb7-07eb2bcaacc7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:26.228660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d72c19-533c-4367-acb7-07eb2bcaacc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:26.230019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d72c19-533c-4367-acb7-07eb2bcaacc7 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:26.234343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 644/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Fri Jun 24 23:39:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:28.355412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62878b0d-3b6b-4a86-9a82-2df5a20d8eb9 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:28.355967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62878b0d-3b6b-4a86-9a82-2df5a20d8eb9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:28.355967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62878b0d-3b6b-4a86-9a82-2df5a20d8eb9 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:28.357386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 649/1293] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:39:28.362471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3399490-ab06-46dd-8f57-2d2e437cc6c8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:28.362880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3399490-ab06-46dd-8f57-2d2e437cc6c8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:28.363028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3399490-ab06-46dd-8f57-2d2e437cc6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:28.379575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f3399490-ab06-46dd-8f57-2d2e437cc6c8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:39:28.380811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f3399490-ab06-46dd-8f57-2d2e437cc6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:39:28.410636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:28.448250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 645/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:39:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:39:28.471031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62878b0d-3b6b-4a86-9a82-2df5a20d8eb9 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:28.501415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:28.570752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62878b0d-3b6b-4a86-9a82-2df5a20d8eb9 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:28.605506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8502ad4-9386-4930-a822-4d31d12cf682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:28.606098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8502ad4-9386-4930-a822-4d31d12cf682 None admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:28.606382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8502ad4-9386-4930-a822-4d31d12cf682 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:28.606660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8502ad4-9386-4930-a822-4d31d12cf682 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:28.607377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8502ad4-9386-4930-a822-4d31d12cf682 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:28.608767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8502ad4-9386-4930-a822-4d31d12cf682 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:28.621149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 650/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Fri Jun 24 23:39:28 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:39:30.669193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec8e8a-7598-4ea9-aa61-e1ff2552f14b None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:30.669722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec8e8a-7598-4ea9-aa61-e1ff2552f14b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:30.669835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec8e8a-7598-4ea9-aa61-e1ff2552f14b None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:30.671550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 646/1296] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:39:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:39:30.677441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00127468-b910-457e-bf52-4b88d5a630ac None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:30.677889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00127468-b910-457e-bf52-4b88d5a630ac None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:30.677984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00127468-b910-457e-bf52-4b88d5a630ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:30.698170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-00127468-b910-457e-bf52-4b88d5a630ac None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:39:30.699435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-00127468-b910-457e-bf52-4b88d5a630ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:39:30.785255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 651/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:39:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:39:30.810751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec8e8a-7598-4ea9-aa61-e1ff2552f14b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:30.909555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec8e8a-7598-4ea9-aa61-e1ff2552f14b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:30.943405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226cf27-5f3e-4048-9b4a-17ac16d8b8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:30.944133 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226cf27-5f3e-4048-9b4a-17ac16d8b8fa None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:30.944424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226cf27-5f3e-4048-9b4a-17ac16d8b8fa None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:30.944742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226cf27-5f3e-4048-9b4a-17ac16d8b8fa None admin] PATH_INFO: `/v3/services/placement` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:30.945403 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226cf27-5f3e-4048-9b4a-17ac16d8b8fa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:30.946800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226cf27-5f3e-4048-9b4a-17ac16d8b8fa None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-1226cf27-5f3e-4048-9b4a-17ac16d8b8fa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:30.958476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:30.959652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:30.961330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 647/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:39:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:30.966692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00127468-b910-457e-bf52-4b88d5a630ac None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:30.985121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:31.063759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00127468-b910-457e-bf52-4b88d5a630ac None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:31.095671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57524f-dfba-4428-a58d-05dd2ffe9bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:31.096315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57524f-dfba-4428-a58d-05dd2ffe9bc4 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:31.096608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57524f-dfba-4428-a58d-05dd2ffe9bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:31.096890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57524f-dfba-4428-a58d-05dd2ffe9bc4 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:31.097595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57524f-dfba-4428-a58d-05dd2ffe9bc4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:31.097900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57524f-dfba-4428-a58d-05dd2ffe9bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:31.099258 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57524f-dfba-4428-a58d-05dd2ffe9bc4 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:31.105050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 652/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Fri Jun 24 23:39:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:31.110856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226cf27-5f3e-4048-9b4a-17ac16d8b8fa None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:31.145129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:31.214474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226cf27-5f3e-4048-9b4a-17ac16d8b8fa None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:31.247132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82560630-7bd4-490a-a4fe-b4f84eb67675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:31.247781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82560630-7bd4-490a-a4fe-b4f84eb67675 None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:31.248066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82560630-7bd4-490a-a4fe-b4f84eb67675 None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:31.248341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82560630-7bd4-490a-a4fe-b4f84eb67675 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:31.249158 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82560630-7bd4-490a-a4fe-b4f84eb67675 None admin] RBAC: Adding query filter params (interface=public, service_id=d1dedeb5937446a1b9a4a421f2859bb8, region_id=RegionOne) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:31.249476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82560630-7bd4-490a-a4fe-b4f84eb67675 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:31.250833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82560630-7bd4-490a-a4fe-b4f84eb67675 None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:31.255719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 648/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Fri Jun 24 23:39:31 2022] GET /identity/v3/endpoints?service_id=d1dedeb5937446a1b9a4a421f2859bb8&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:31.261891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57524f-dfba-4428-a58d-05dd2ffe9bc4 None admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:31.341541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:31.360312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57524f-dfba-4428-a58d-05dd2ffe9bc4 None admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:31.393531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82dd6d-8319-4ff1-8b19-4981908039f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:31.394191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82dd6d-8319-4ff1-8b19-4981908039f3 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:31.394476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82dd6d-8319-4ff1-8b19-4981908039f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:31.394756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82dd6d-8319-4ff1-8b19-4981908039f3 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:31.395687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82dd6d-8319-4ff1-8b19-4981908039f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:31.397095 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82dd6d-8319-4ff1-8b19-4981908039f3 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:31.404073 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 653/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Fri Jun 24 23:39:31 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:39:33.389938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf810ca0-03c7-45ac-9322-a3609bc58d3c None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:33.390459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf810ca0-03c7-45ac-9322-a3609bc58d3c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:33.390552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf810ca0-03c7-45ac-9322-a3609bc58d3c None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:33.391838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 649/1302] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:39:33.397023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cb60d-6c06-4862-a493-a47f28e04366 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:33.397420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cb60d-6c06-4862-a493-a47f28e04366 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:33.397575 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1cb60d-6c06-4862-a493-a47f28e04366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:33.414420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-5e1cb60d-6c06-4862-a493-a47f28e04366 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:39:33.416197 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5e1cb60d-6c06-4862-a493-a47f28e04366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:39:33.479052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 654/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:39:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 23:39:33.501478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf810ca0-03c7-45ac-9322-a3609bc58d3c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:33.593862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:33.600903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf810ca0-03c7-45ac-9322-a3609bc58d3c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:33.633756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb716ae5-f4a4-4d0d-b3c0-364d3fc0970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:33.634538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb716ae5-f4a4-4d0d-b3c0-364d3fc0970e None admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:33.634823 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb716ae5-f4a4-4d0d-b3c0-364d3fc0970e None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:33.635089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb716ae5-f4a4-4d0d-b3c0-364d3fc0970e None admin] PATH_INFO: `/v3/services/placement` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:33.635734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb716ae5-f4a4-4d0d-b3c0-364d3fc0970e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:33.637122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb716ae5-f4a4-4d0d-b3c0-364d3fc0970e None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application [None req-fb716ae5-f4a4-4d0d-b3c0-364d3fc0970e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 23:39:33.649614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 23:39:33.650731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application with Lock( Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self._enter() Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application return self.creator() Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application created_value = creator( Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: ERROR keystone.server.flask.application Jun 24 23:39:33.651576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 650/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Fri Jun 24 23:39:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 23:39:33.656599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cb60d-6c06-4862-a493-a47f28e04366 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:33.729426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:33.754787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1cb60d-6c06-4862-a493-a47f28e04366 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:33.793685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce075d8-f754-4725-920a-413b486eaf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:33.794351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce075d8-f754-4725-920a-413b486eaf82 None admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:33.794635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce075d8-f754-4725-920a-413b486eaf82 None admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:33.794912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce075d8-f754-4725-920a-413b486eaf82 None admin] PATH_INFO: `/v3/services` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:33.795625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce075d8-f754-4725-920a-413b486eaf82 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:39:33.795937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce075d8-f754-4725-920a-413b486eaf82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:33.797338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce075d8-f754-4725-920a-413b486eaf82 None admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:33.802408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 655/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Fri Jun 24 23:39:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:39:33.809043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb716ae5-f4a4-4d0d-b3c0-364d3fc0970e None admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:39:33.904556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb716ae5-f4a4-4d0d-b3c0-364d3fc0970e None admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:39:33.937041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6937f1-43cf-4cb0-b3fb-7cd61943dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:39:33.937689 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6937f1-43cf-4cb0-b3fb-7cd61943dc0b None admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:39:33.937965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6937f1-43cf-4cb0-b3fb-7cd61943dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:39:33.938234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6937f1-43cf-4cb0-b3fb-7cd61943dc0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:39:33.938968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6937f1-43cf-4cb0-b3fb-7cd61943dc0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:39:33.940315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6937f1-43cf-4cb0-b3fb-7cd61943dc0b None admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:39:33.958212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 651/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Fri Jun 24 23:39:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:39:43.799553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=30 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:39:43.953786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:37.065225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5c72-f52d-46c2-9f72-8e319d071be4 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:37.065807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5c72-f52d-46c2-9f72-8e319d071be4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:37.065807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5c72-f52d-46c2-9f72-8e319d071be4 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:37.071117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 656/1307] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:40:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:37.072722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78fd31-d902-409f-ae61-e86ad5b137c3 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:37.073168 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78fd31-d902-409f-ae61-e86ad5b137c3 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:37.073269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78fd31-d902-409f-ae61-e86ad5b137c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:37.188288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-9b78fd31-d902-409f-ae61-e86ad5b137c3 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:37.190526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9b78fd31-d902-409f-ae61-e86ad5b137c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:37.363572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 652/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:40:37.439477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d909-9ca9-40e6-9feb-e27f84171d9e None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:37.439743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d909-9ca9-40e6-9feb-e27f84171d9e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:37.440030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d909-9ca9-40e6-9feb-e27f84171d9e None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:37.441602 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 657/1309] 10.222.0.47 () {54 vars in 964 bytes} [Fri Jun 24 23:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:37.448575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbadb6-9d80-4f85-bce8-8a64d617acf8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:37.448895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbadb6-9d80-4f85-bce8-8a64d617acf8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:37.449178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bbadb6-9d80-4f85-bce8-8a64d617acf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:37.562947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-56bbadb6-9d80-4f85-bce8-8a64d617acf8 None None] MFA Rules not processed for user `e38a311f2b614a3598f6b51b3a235178`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:37.564236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-56bbadb6-9d80-4f85-bce8-8a64d617acf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:37.595947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:37.667282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 653/1310] 10.222.0.47 () {58 vars in 1061 bytes} [Fri Jun 24 23:40:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:40:37.677186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d909-9ca9-40e6-9feb-e27f84171d9e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:40:37.678960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f557d909-9ca9-40e6-9feb-e27f84171d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:37.798832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d909-9ca9-40e6-9feb-e27f84171d9e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:40:37.881382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11a0a3-510b-4ae6-b872-e04d4a201166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:40:37.882051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11a0a3-510b-4ae6-b872-e04d4a201166 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:37.882342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11a0a3-510b-4ae6-b872-e04d4a201166 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:37.882623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11a0a3-510b-4ae6-b872-e04d4a201166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:37.884288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-af11a0a3-510b-4ae6-b872-e04d4a201166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:37.944086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:38.004542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11a0a3-510b-4ae6-b872-e04d4a201166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:40:38.006787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11a0a3-510b-4ae6-b872-e04d4a201166 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:40:38.031596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 658/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:40:40.508827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38979e-81aa-4edc-b73f-d3d8fdda0c6e None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:40.509364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38979e-81aa-4edc-b73f-d3d8fdda0c6e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:40.509496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38979e-81aa-4edc-b73f-d3d8fdda0c6e None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:40.510758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 654/1312] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:40.515888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb4668-5675-42f5-b1b7-3dcaf70beb17 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:40.516295 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb4668-5675-42f5-b1b7-3dcaf70beb17 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:40.516441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb4668-5675-42f5-b1b7-3dcaf70beb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:40.585954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3ccb4668-5675-42f5-b1b7-3dcaf70beb17 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:40.587414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3ccb4668-5675-42f5-b1b7-3dcaf70beb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:40.685180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 659/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:40:40.725595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38979e-81aa-4edc-b73f-d3d8fdda0c6e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:40:40.727164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0d38979e-81aa-4edc-b73f-d3d8fdda0c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:40.844598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38979e-81aa-4edc-b73f-d3d8fdda0c6e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:40:40.915073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:40.917795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf79b6-d3f1-482e-b9cd-f6d09d035c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:40:40.918451 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf79b6-d3f1-482e-b9cd-f6d09d035c6e service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:40.918927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf79b6-d3f1-482e-b9cd-f6d09d035c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:40.919360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf79b6-d3f1-482e-b9cd-f6d09d035c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:40.921262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-fddf79b6-d3f1-482e-b9cd-f6d09d035c6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:41.033766 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf79b6-d3f1-482e-b9cd-f6d09d035c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:40:41.035242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf79b6-d3f1-482e-b9cd-f6d09d035c6e service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:40:41.054015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 655/1314] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:40:43.388424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4905e0-6169-4fba-89d5-c26509fe7f86 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:43.389014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4905e0-6169-4fba-89d5-c26509fe7f86 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:43.389014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4905e0-6169-4fba-89d5-c26509fe7f86 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:43.390391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 660/1315] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:43.396443 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d081c49-af70-4595-8afa-781cb77716de None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:43.396896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d081c49-af70-4595-8afa-781cb77716de None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:43.397008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d081c49-af70-4595-8afa-781cb77716de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:43.463290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-2d081c49-af70-4595-8afa-781cb77716de None None] MFA Rules not processed for user `572c94b6640b44cc8fb0508eda65e95d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:43.464511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2d081c49-af70-4595-8afa-781cb77716de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:43.515067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:43.559346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 656/1316] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:40:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:40:43.611254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da81233-9bd9-4eee-9558-45bb74d69c01 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:43.611534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da81233-9bd9-4eee-9558-45bb74d69c01 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:43.611811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da81233-9bd9-4eee-9558-45bb74d69c01 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:43.613231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 661/1317] 10.222.0.47 () {54 vars in 964 bytes} [Fri Jun 24 23:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:43.619412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1381d0f-6de1-46d3-8eee-f8ced08f2e45 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:43.619702 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1381d0f-6de1-46d3-8eee-f8ced08f2e45 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:43.619979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1381d0f-6de1-46d3-8eee-f8ced08f2e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:43.701910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c1381d0f-6de1-46d3-8eee-f8ced08f2e45 None None] MFA Rules not processed for user `e38a311f2b614a3598f6b51b3a235178`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:43.703059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c1381d0f-6de1-46d3-8eee-f8ced08f2e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:43.780598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 657/1318] 10.222.0.47 () {58 vars in 1061 bytes} [Fri Jun 24 23:40:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:40:43.789777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da81233-9bd9-4eee-9558-45bb74d69c01 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:40:43.791346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3da81233-9bd9-4eee-9558-45bb74d69c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:43.800923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:43.893397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da81233-9bd9-4eee-9558-45bb74d69c01 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:40:43.964908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb9b77-7bc5-497f-8778-12d2fbe55c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:40:43.965554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb9b77-7bc5-497f-8778-12d2fbe55c98 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:43.965836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb9b77-7bc5-497f-8778-12d2fbe55c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:43.966119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb9b77-7bc5-497f-8778-12d2fbe55c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:43.967790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e7fb9b77-7bc5-497f-8778-12d2fbe55c98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:44.029449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:44.075265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb9b77-7bc5-497f-8778-12d2fbe55c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:40:44.076744 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb9b77-7bc5-497f-8778-12d2fbe55c98 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:40:44.094058 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 662/1319] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:40:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:40:47.039317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e779cb-5fd5-4192-b6a2-c2863b444455 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:47.039852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e779cb-5fd5-4192-b6a2-c2863b444455 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:47.039852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e779cb-5fd5-4192-b6a2-c2863b444455 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:47.041255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 658/1320] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:47.046276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7791829-4f4f-4082-8259-2c513ffd3164 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:47.046662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7791829-4f4f-4082-8259-2c513ffd3164 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:47.046841 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7791829-4f4f-4082-8259-2c513ffd3164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:47.110031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f7791829-4f4f-4082-8259-2c513ffd3164 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:47.111225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f7791829-4f4f-4082-8259-2c513ffd3164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:47.204932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 663/1321] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:40:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:40:47.244383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e779cb-5fd5-4192-b6a2-c2863b444455 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:40:47.245926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-18e779cb-5fd5-4192-b6a2-c2863b444455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:47.295429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:47.349436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e779cb-5fd5-4192-b6a2-c2863b444455 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:40:47.420388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883843f8-c666-4ba6-bcba-2de35f367de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:40:47.421058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883843f8-c666-4ba6-bcba-2de35f367de2 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:47.421338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883843f8-c666-4ba6-bcba-2de35f367de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:47.421611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883843f8-c666-4ba6-bcba-2de35f367de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:47.423226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-883843f8-c666-4ba6-bcba-2de35f367de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:47.517455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:47.531150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883843f8-c666-4ba6-bcba-2de35f367de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:40:47.532596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883843f8-c666-4ba6-bcba-2de35f367de2 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:40:47.550738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 659/1322] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:40:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:40:50.327044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26658-7c09-429a-8c21-1eff96f9df71 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:50.327570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26658-7c09-429a-8c21-1eff96f9df71 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:50.327570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26658-7c09-429a-8c21-1eff96f9df71 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:50.328904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 664/1323] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:50.335784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93c363-bdc9-4590-b7ba-08f44ca3c386 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:50.336225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93c363-bdc9-4590-b7ba-08f44ca3c386 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:50.336339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93c363-bdc9-4590-b7ba-08f44ca3c386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:50.401961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-3f93c363-bdc9-4590-b7ba-08f44ca3c386 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:50.403156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3f93c363-bdc9-4590-b7ba-08f44ca3c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:50.509509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 660/1324] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:40:50.547185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26658-7c09-429a-8c21-1eff96f9df71 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:40:50.548907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d8f26658-7c09-429a-8c21-1eff96f9df71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:50.569235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:50.662088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26658-7c09-429a-8c21-1eff96f9df71 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:40:50.736279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51449b3-f74e-4919-9aec-6bc31e123349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:40:50.736963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51449b3-f74e-4919-9aec-6bc31e123349 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:50.737246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51449b3-f74e-4919-9aec-6bc31e123349 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:50.737519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51449b3-f74e-4919-9aec-6bc31e123349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:50.740145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b51449b3-f74e-4919-9aec-6bc31e123349 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:50.824638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:50.864275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51449b3-f74e-4919-9aec-6bc31e123349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:40:50.865776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51449b3-f74e-4919-9aec-6bc31e123349 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:40:50.884994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 665/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:40:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:40:53.004918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8dfb-05dc-4547-87c4-b05b1aaa43a9 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:53.005473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8dfb-05dc-4547-87c4-b05b1aaa43a9 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:53.005473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8dfb-05dc-4547-87c4-b05b1aaa43a9 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:53.006780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 661/1326] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:53.012448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47db38c-b268-4db2-8b26-4d4cf9f89465 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:53.012929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47db38c-b268-4db2-8b26-4d4cf9f89465 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:53.013170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47db38c-b268-4db2-8b26-4d4cf9f89465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:53.078711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-e47db38c-b268-4db2-8b26-4d4cf9f89465 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:53.079920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e47db38c-b268-4db2-8b26-4d4cf9f89465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:53.186532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 666/1327] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:40:53.222876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8dfb-05dc-4547-87c4-b05b1aaa43a9 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:40:53.224432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-971c8dfb-05dc-4547-87c4-b05b1aaa43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:53.284839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:53.329291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8dfb-05dc-4547-87c4-b05b1aaa43a9 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:40:53.405012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f96887-ef73-482c-adc3-98fa4abb97a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:40:53.405689 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f96887-ef73-482c-adc3-98fa4abb97a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:53.405972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f96887-ef73-482c-adc3-98fa4abb97a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:53.406251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f96887-ef73-482c-adc3-98fa4abb97a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:53.408015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-89f96887-ef73-482c-adc3-98fa4abb97a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:53.518075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:53.521600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f96887-ef73-482c-adc3-98fa4abb97a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:40:53.523491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f96887-ef73-482c-adc3-98fa4abb97a0 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:40:53.541990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 662/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:40:55.657560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eede20c-5fd9-4f16-bda4-68b550fed5b0 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:55.658117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eede20c-5fd9-4f16-bda4-68b550fed5b0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:55.658117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eede20c-5fd9-4f16-bda4-68b550fed5b0 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:55.659483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 667/1329] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:55.665228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44938be8-bdec-4479-b97e-c1e80d847e3a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:55.665655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44938be8-bdec-4479-b97e-c1e80d847e3a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:55.665926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44938be8-bdec-4479-b97e-c1e80d847e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:55.735240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-44938be8-bdec-4479-b97e-c1e80d847e3a None None] MFA Rules not processed for user `572c94b6640b44cc8fb0508eda65e95d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:55.736383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-44938be8-bdec-4479-b97e-c1e80d847e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:55.837410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 663/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:40:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:40:55.874892 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eede20c-5fd9-4f16-bda4-68b550fed5b0 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:40:55.876427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0eede20c-5fd9-4f16-bda4-68b550fed5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:55.907938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:55.985570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eede20c-5fd9-4f16-bda4-68b550fed5b0 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:40:56.061704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731d77c-e01c-418b-949c-f5cfadbaaab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:40:56.062361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731d77c-e01c-418b-949c-f5cfadbaaab4 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:56.062649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731d77c-e01c-418b-949c-f5cfadbaaab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:56.062928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731d77c-e01c-418b-949c-f5cfadbaaab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:56.064600 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3731d77c-e01c-418b-949c-f5cfadbaaab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:56.142078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:56.181291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731d77c-e01c-418b-949c-f5cfadbaaab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:40:56.182778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731d77c-e01c-418b-949c-f5cfadbaaab4 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:40:56.207441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 668/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:40:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:40:58.532917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e8b75-31a5-467b-8093-081a7056b3d3 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:58.533424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e8b75-31a5-467b-8093-081a7056b3d3 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:58.533424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e8b75-31a5-467b-8093-081a7056b3d3 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:58.534931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 664/1332] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:40:58.540204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed9db8-0149-4257-b497-7ba23d0bb641 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:58.540612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed9db8-0149-4257-b497-7ba23d0bb641 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:58.540801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed9db8-0149-4257-b497-7ba23d0bb641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:58.604275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6eed9db8-0149-4257-b497-7ba23d0bb641 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:40:58.605558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6eed9db8-0149-4257-b497-7ba23d0bb641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:58.702216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 669/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:40:58.738338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e8b75-31a5-467b-8093-081a7056b3d3 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:40:58.739903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8f9e8b75-31a5-467b-8093-081a7056b3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:58.817913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:58.852321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e8b75-31a5-467b-8093-081a7056b3d3 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:40:58.924653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366635d5-7eae-4c49-adaa-d512d8159d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:40:58.925792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366635d5-7eae-4c49-adaa-d512d8159d61 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:40:58.926075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366635d5-7eae-4c49-adaa-d512d8159d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:40:58.926350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366635d5-7eae-4c49-adaa-d512d8159d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:40:58.928004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-366635d5-7eae-4c49-adaa-d512d8159d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:40:59.035349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366635d5-7eae-4c49-adaa-d512d8159d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:40:59.036820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366635d5-7eae-4c49-adaa-d512d8159d61 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:40:59.046673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:40:59.056919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 665/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:01.153851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae051ac7-0621-49eb-a1a0-878e8fe4c6c6 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:01.154361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae051ac7-0621-49eb-a1a0-878e8fe4c6c6 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:01.154466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae051ac7-0621-49eb-a1a0-878e8fe4c6c6 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:01.156099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 670/1335] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:01.161343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb9bea-5578-47de-b0bf-3adc3a9d2c0b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:01.161748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb9bea-5578-47de-b0bf-3adc3a9d2c0b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:01.161902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb9bea-5578-47de-b0bf-3adc3a9d2c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:01.225674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-5beb9bea-5578-47de-b0bf-3adc3a9d2c0b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:01.226800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5beb9bea-5578-47de-b0bf-3adc3a9d2c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:01.322097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 666/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:01.361469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae051ac7-0621-49eb-a1a0-878e8fe4c6c6 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:01.363144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ae051ac7-0621-49eb-a1a0-878e8fe4c6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:01.403438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:01.465763 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae051ac7-0621-49eb-a1a0-878e8fe4c6c6 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:01.536973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d76169-0d51-4f81-89ae-c25b95de6d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:01.537547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d76169-0d51-4f81-89ae-c25b95de6d35 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:01.537830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d76169-0d51-4f81-89ae-c25b95de6d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:01.538109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d76169-0d51-4f81-89ae-c25b95de6d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:01.539743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-97d76169-0d51-4f81-89ae-c25b95de6d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:01.619600 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:01.646712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d76169-0d51-4f81-89ae-c25b95de6d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:01.648253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d76169-0d51-4f81-89ae-c25b95de6d35 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:01.666990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 671/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:04.163857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b34d4-d5ef-493c-b6ff-81d58c8041f5 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:04.164384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b34d4-d5ef-493c-b6ff-81d58c8041f5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:04.164507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b34d4-d5ef-493c-b6ff-81d58c8041f5 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:04.165801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 667/1338] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:04.170973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c01cfc-2136-48e3-9b1f-762db6465c95 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:04.171350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c01cfc-2136-48e3-9b1f-762db6465c95 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:04.171532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c01cfc-2136-48e3-9b1f-762db6465c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:04.237799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-e9c01cfc-2136-48e3-9b1f-762db6465c95 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:04.239100 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e9c01cfc-2136-48e3-9b1f-762db6465c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:04.334050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 672/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:04.369722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b34d4-d5ef-493c-b6ff-81d58c8041f5 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:04.371242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4d9b34d4-d5ef-493c-b6ff-81d58c8041f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:04.446170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:04.474032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b34d4-d5ef-493c-b6ff-81d58c8041f5 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:04.549881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496eaee5-ff1e-4120-83ad-3a44a5b28589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:04.551042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496eaee5-ff1e-4120-83ad-3a44a5b28589 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:04.551351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496eaee5-ff1e-4120-83ad-3a44a5b28589 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:04.551630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496eaee5-ff1e-4120-83ad-3a44a5b28589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:04.553349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-496eaee5-ff1e-4120-83ad-3a44a5b28589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:04.661233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496eaee5-ff1e-4120-83ad-3a44a5b28589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:04.662881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496eaee5-ff1e-4120-83ad-3a44a5b28589 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:04.683167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 668/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:08.839893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e2bbf-1541-4937-a89e-50b5731ff04e None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:08.840379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e2bbf-1541-4937-a89e-50b5731ff04e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:08.840479 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e2bbf-1541-4937-a89e-50b5731ff04e None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:08.844532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 673/1341] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:08.848331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be41a53-7176-4665-a1e7-a67adb0aff0d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:08.848824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be41a53-7176-4665-a1e7-a67adb0aff0d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:08.849204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be41a53-7176-4665-a1e7-a67adb0aff0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:08.855856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:08.923105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-9be41a53-7176-4665-a1e7-a67adb0aff0d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:08.924412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9be41a53-7176-4665-a1e7-a67adb0aff0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:09.038946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 669/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:09.076279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e2bbf-1541-4937-a89e-50b5731ff04e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:09.078497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-aa6e2bbf-1541-4937-a89e-50b5731ff04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:09.128374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:09.174207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e2bbf-1541-4937-a89e-50b5731ff04e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:09.177960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b227b5-dfda-497a-960e-4ac148b59431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:09.178605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b227b5-dfda-497a-960e-4ac148b59431 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:09.178888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b227b5-dfda-497a-960e-4ac148b59431 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:09.179162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b227b5-dfda-497a-960e-4ac148b59431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:09.188177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b227b5-dfda-497a-960e-4ac148b59431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:09.189632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b227b5-dfda-497a-960e-4ac148b59431 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:09.199682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 674/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:11.918512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbe03f-e3bf-4ea9-b6ad-b988b86f1ac1 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:11.919074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbe03f-e3bf-4ea9-b6ad-b988b86f1ac1 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:11.919074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbe03f-e3bf-4ea9-b6ad-b988b86f1ac1 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:11.920468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 670/1344] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:11.926427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16851e-dcff-4659-8644-befb396116a1 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:11.926854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16851e-dcff-4659-8644-befb396116a1 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:11.926990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16851e-dcff-4659-8644-befb396116a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:11.958478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3a16851e-dcff-4659-8644-befb396116a1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:11.959782 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3a16851e-dcff-4659-8644-befb396116a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:12.045702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 675/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:12.084574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbe03f-e3bf-4ea9-b6ad-b988b86f1ac1 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:12.086440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e9dbe03f-e3bf-4ea9-b6ad-b988b86f1ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:12.101140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbe03f-e3bf-4ea9-b6ad-b988b86f1ac1 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:12.104751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4a2c1-9bbb-40ee-8d37-c2c0683c2297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:12.105412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4a2c1-9bbb-40ee-8d37-c2c0683c2297 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:12.105695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4a2c1-9bbb-40ee-8d37-c2c0683c2297 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:12.105978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4a2c1-9bbb-40ee-8d37-c2c0683c2297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:12.116367 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4a2c1-9bbb-40ee-8d37-c2c0683c2297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:12.117806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4a2c1-9bbb-40ee-8d37-c2c0683c2297 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:12.127596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 671/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:16.392033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288df8e9-df00-4072-ab58-b0343250bad7 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:16.392531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288df8e9-df00-4072-ab58-b0343250bad7 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:16.392643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288df8e9-df00-4072-ab58-b0343250bad7 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:16.394863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 676/1347] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:16.400438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685df9a-4270-4a26-a3fa-dd7f1dc54d84 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:16.400924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685df9a-4270-4a26-a3fa-dd7f1dc54d84 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:16.401051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685df9a-4270-4a26-a3fa-dd7f1dc54d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:16.419756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f685df9a-4270-4a26-a3fa-dd7f1dc54d84 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:16.420980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f685df9a-4270-4a26-a3fa-dd7f1dc54d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:16.428970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 672/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:16.467117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288df8e9-df00-4072-ab58-b0343250bad7 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:16.478434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288df8e9-df00-4072-ab58-b0343250bad7 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:16.481966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add08d7-42b3-4ff3-b9c8-154c132f4a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:16.482626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add08d7-42b3-4ff3-b9c8-154c132f4a1e service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:16.482915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add08d7-42b3-4ff3-b9c8-154c132f4a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:16.483194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add08d7-42b3-4ff3-b9c8-154c132f4a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:16.492845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add08d7-42b3-4ff3-b9c8-154c132f4a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:16.494360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add08d7-42b3-4ff3-b9c8-154c132f4a1e service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:16.504494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 677/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:21.507836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163f34b-b781-4959-b90a-d59706efa074 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:21.508330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163f34b-b781-4959-b90a-d59706efa074 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:21.508536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163f34b-b781-4959-b90a-d59706efa074 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:21.510355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 673/1350] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:21.515992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcebc72-63b8-4ffb-98df-3460a3b4fc00 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:21.516422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcebc72-63b8-4ffb-98df-3460a3b4fc00 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:21.516559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcebc72-63b8-4ffb-98df-3460a3b4fc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:21.631069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0dcebc72-63b8-4ffb-98df-3460a3b4fc00 None None] MFA Rules not processed for user `a243d444b7ae487cb0331523d4404d19`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:21.632367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0dcebc72-63b8-4ffb-98df-3460a3b4fc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:21.665174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:21.721485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 678/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:41:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:25.873644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d93896-44c0-4b08-875d-d381cb315a04 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:25.874214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d93896-44c0-4b08-875d-d381cb315a04 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:25.874214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d93896-44c0-4b08-875d-d381cb315a04 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:25.875560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 674/1352] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:25.881544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fa1c4-38c9-4ebf-a47f-c94bf4ad320c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:25.881974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fa1c4-38c9-4ebf-a47f-c94bf4ad320c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:25.882139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845fa1c4-38c9-4ebf-a47f-c94bf4ad320c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:25.901200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-845fa1c4-38c9-4ebf-a47f-c94bf4ad320c None None] MFA Rules not processed for user `a243d444b7ae487cb0331523d4404d19`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:25.902516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-845fa1c4-38c9-4ebf-a47f-c94bf4ad320c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:25.911451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 679/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:41:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:26.413471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=72 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:27.662055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec8887-b73d-488f-acdf-ee9caa275a07 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:27.662677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec8887-b73d-488f-acdf-ee9caa275a07 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:27.662677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec8887-b73d-488f-acdf-ee9caa275a07 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:27.663960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 675/1354] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:27.669234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078df666-3a33-4c67-b355-cb2dd55b1037 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:27.669678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078df666-3a33-4c67-b355-cb2dd55b1037 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:27.669770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078df666-3a33-4c67-b355-cb2dd55b1037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:27.689876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-078df666-3a33-4c67-b355-cb2dd55b1037 None None] MFA Rules not processed for user `a243d444b7ae487cb0331523d4404d19`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:27.691076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-078df666-3a33-4c67-b355-cb2dd55b1037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:27.699869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 680/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:41:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:31.321701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe710e9-97e1-457c-8ef4-426c1c29959b None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:31.322193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe710e9-97e1-457c-8ef4-426c1c29959b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:31.322193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe710e9-97e1-457c-8ef4-426c1c29959b None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:31.323705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 676/1356] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:31.329689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dafea-37d1-4e9a-a1ad-13e16853d653 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:31.330179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dafea-37d1-4e9a-a1ad-13e16853d653 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:31.330399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dafea-37d1-4e9a-a1ad-13e16853d653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:31.355260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-8e2dafea-37d1-4e9a-a1ad-13e16853d653 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:31.357003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8e2dafea-37d1-4e9a-a1ad-13e16853d653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:31.365388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 681/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:31.436391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ba4e-f93a-4f74-9afa-2bd31eef91e9 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:31.437276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ba4e-f93a-4f74-9afa-2bd31eef91e9 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:31.437827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ba4e-f93a-4f74-9afa-2bd31eef91e9 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:31.440552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 677/1358] 10.222.0.47 () {54 vars in 961 bytes} [Fri Jun 24 23:41:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:31.455608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234c902-427e-4a6d-8961-14d5c93b9d44 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:31.456358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234c902-427e-4a6d-8961-14d5c93b9d44 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:31.456719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234c902-427e-4a6d-8961-14d5c93b9d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:31.513743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2234c902-427e-4a6d-8961-14d5c93b9d44 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:31.514940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2234c902-427e-4a6d-8961-14d5c93b9d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:31.607483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 682/1359] 10.222.0.47 () {58 vars in 1058 bytes} [Fri Jun 24 23:41:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:31.615831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38ba4e-f93a-4f74-9afa-2bd31eef91e9 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:31.631157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38ba4e-f93a-4f74-9afa-2bd31eef91e9 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:31.636665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b4d5d-36f6-4f7d-a001-efe25532b5c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:31.637391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4d5d-36f6-4f7d-a001-efe25532b5c0 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:31.637743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4d5d-36f6-4f7d-a001-efe25532b5c0 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:31.638089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4d5d-36f6-4f7d-a001-efe25532b5c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:31.648043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b4d5d-36f6-4f7d-a001-efe25532b5c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:31.649754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b4d5d-36f6-4f7d-a001-efe25532b5c0 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:31.660812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 678/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:33.923130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45648f-6048-453a-95d1-7fa4d5d1383e None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:33.924846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45648f-6048-453a-95d1-7fa4d5d1383e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:33.924846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45648f-6048-453a-95d1-7fa4d5d1383e None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:33.929617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 683/1361] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:33.931224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf143a-4b99-48ed-b95d-b9631573a161 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:33.932223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf143a-4b99-48ed-b95d-b9631573a161 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:33.932710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf143a-4b99-48ed-b95d-b9631573a161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:33.966307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c6bf143a-4b99-48ed-b95d-b9631573a161 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:33.967594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c6bf143a-4b99-48ed-b95d-b9631573a161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:33.980679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 679/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:34.023548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45648f-6048-453a-95d1-7fa4d5d1383e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:34.039049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45648f-6048-453a-95d1-7fa4d5d1383e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:34.045222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c2bb5-7c3a-4fb9-a4a5-f8229da90dc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:34.046094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c2bb5-7c3a-4fb9-a4a5-f8229da90dc9 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:34.046573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c2bb5-7c3a-4fb9-a4a5-f8229da90dc9 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:34.047135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c2bb5-7c3a-4fb9-a4a5-f8229da90dc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:34.057247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c2bb5-7c3a-4fb9-a4a5-f8229da90dc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:34.059074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c2bb5-7c3a-4fb9-a4a5-f8229da90dc9 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:34.069835 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 684/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:36.479447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f13724-727c-4339-89fb-39de73861bf2 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:36.479934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f13724-727c-4339-89fb-39de73861bf2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:36.479934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f13724-727c-4339-89fb-39de73861bf2 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:36.481644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 680/1364] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:36.487775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd2e6b-8649-4dce-9c2a-76e163646a44 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:36.488558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd2e6b-8649-4dce-9c2a-76e163646a44 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:36.489044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd2e6b-8649-4dce-9c2a-76e163646a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:36.512327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-41bd2e6b-8649-4dce-9c2a-76e163646a44 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:36.513569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-41bd2e6b-8649-4dce-9c2a-76e163646a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:36.521891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 685/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:36.567379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2ab86-23bc-45fb-896f-a2b8d685bca6 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:36.567666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2ab86-23bc-45fb-896f-a2b8d685bca6 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:36.567949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2ab86-23bc-45fb-896f-a2b8d685bca6 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:36.569333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 681/1366] 10.222.0.47 () {54 vars in 961 bytes} [Fri Jun 24 23:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:36.574619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd93d86-fa75-45de-9447-fe8aafb9d086 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:36.574914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd93d86-fa75-45de-9447-fe8aafb9d086 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:36.575196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd93d86-fa75-45de-9447-fe8aafb9d086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:36.593150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9bd93d86-fa75-45de-9447-fe8aafb9d086 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:36.594339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9bd93d86-fa75-45de-9447-fe8aafb9d086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:36.604034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 686/1367] 10.222.0.47 () {58 vars in 1058 bytes} [Fri Jun 24 23:41:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:41:36.612624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2ab86-23bc-45fb-896f-a2b8d685bca6 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:36.627395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2ab86-23bc-45fb-896f-a2b8d685bca6 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:36.632842 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd092576-aa8d-401c-a92a-6f5ac73c9315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:36.633474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd092576-aa8d-401c-a92a-6f5ac73c9315 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:36.633756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd092576-aa8d-401c-a92a-6f5ac73c9315 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:36.634035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd092576-aa8d-401c-a92a-6f5ac73c9315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:36.642628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd092576-aa8d-401c-a92a-6f5ac73c9315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:36.644448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd092576-aa8d-401c-a92a-6f5ac73c9315 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:36.654353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 682/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:38.778480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2345b2-7f7c-41a6-b02f-f093a521f838 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:38.779062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2345b2-7f7c-41a6-b02f-f093a521f838 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:38.779354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2345b2-7f7c-41a6-b02f-f093a521f838 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:38.780988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 687/1369] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:38.786200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bf820-ece6-4d65-a12a-0c10a77ebf7d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:38.786634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bf820-ece6-4d65-a12a-0c10a77ebf7d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:38.786982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bf820-ece6-4d65-a12a-0c10a77ebf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:38.805580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-610bf820-ece6-4d65-a12a-0c10a77ebf7d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:38.806978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-610bf820-ece6-4d65-a12a-0c10a77ebf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:38.816142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 683/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:38.863129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2345b2-7f7c-41a6-b02f-f093a521f838 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:38.880499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2345b2-7f7c-41a6-b02f-f093a521f838 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:38.886077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cba70d-3c9b-424b-a3ee-b43050b91ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:38.886921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cba70d-3c9b-424b-a3ee-b43050b91ce6 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:38.887214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cba70d-3c9b-424b-a3ee-b43050b91ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:38.887495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cba70d-3c9b-424b-a3ee-b43050b91ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:38.896485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cba70d-3c9b-424b-a3ee-b43050b91ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:38.898128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cba70d-3c9b-424b-a3ee-b43050b91ce6 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:38.910643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 688/1371] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:39.573705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f80794-e823-4326-a3e2-4e678de4eb70 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:39.574249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f80794-e823-4326-a3e2-4e678de4eb70 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:39.574706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f80794-e823-4326-a3e2-4e678de4eb70 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:39.576241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 684/1372] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:39.581882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b0e54-f4df-40c8-a390-53729d46c224 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:39.582368 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b0e54-f4df-40c8-a390-53729d46c224 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:39.582799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b0e54-f4df-40c8-a390-53729d46c224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:39.601557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c07b0e54-f4df-40c8-a390-53729d46c224 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:39.603016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c07b0e54-f4df-40c8-a390-53729d46c224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:39.611688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 689/1373] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:39.923123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cc8f1-8710-4ceb-8a65-de06a7aab774 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:39.923593 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cc8f1-8710-4ceb-8a65-de06a7aab774 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:39.923906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cc8f1-8710-4ceb-8a65-de06a7aab774 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:39.925520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 685/1374] 10.222.0.47 () {54 vars in 963 bytes} [Fri Jun 24 23:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:39.930471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd89c8-2000-4aac-9737-ae1c7c4e7100 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:39.930892 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd89c8-2000-4aac-9737-ae1c7c4e7100 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:39.931094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd89c8-2000-4aac-9737-ae1c7c4e7100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:39.937803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:39.992077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b7fd89c8-2000-4aac-9737-ae1c7c4e7100 None None] MFA Rules not processed for user `fb004ddd208849f6a5316c19e9a916aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:39.993477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b7fd89c8-2000-4aac-9737-ae1c7c4e7100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:40.059216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 690/1375] 10.222.0.47 () {58 vars in 1060 bytes} [Fri Jun 24 23:41:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:40.067080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cc8f1-8710-4ceb-8a65-de06a7aab774 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:40.078179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cc8f1-8710-4ceb-8a65-de06a7aab774 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:40.082699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9fdf6-a010-4536-bda9-856551ef9d2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:40.083517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9fdf6-a010-4536-bda9-856551ef9d2a service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:40.083912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9fdf6-a010-4536-bda9-856551ef9d2a service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:40.084324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9fdf6-a010-4536-bda9-856551ef9d2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:40.095477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9fdf6-a010-4536-bda9-856551ef9d2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:40.097243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9fdf6-a010-4536-bda9-856551ef9d2a service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:40.108474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 686/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:41.027482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66cd9b-1175-4b98-b4a3-b5e39d53b232 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:41.027482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66cd9b-1175-4b98-b4a3-b5e39d53b232 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:41.027482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66cd9b-1175-4b98-b4a3-b5e39d53b232 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:41.028848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 691/1377] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:41.034004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776f1db-8da9-4360-9e27-0dec17b97ccc None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:41.034519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776f1db-8da9-4360-9e27-0dec17b97ccc None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:41.034614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776f1db-8da9-4360-9e27-0dec17b97ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:41.053744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-7776f1db-8da9-4360-9e27-0dec17b97ccc None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:41.055035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7776f1db-8da9-4360-9e27-0dec17b97ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:41.063459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 687/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:41:41.108490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66cd9b-1175-4b98-b4a3-b5e39d53b232 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:41.129065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66cd9b-1175-4b98-b4a3-b5e39d53b232 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:41.134748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484cb2b-1814-4c46-9e26-8d1b89e5209a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:41.135397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484cb2b-1814-4c46-9e26-8d1b89e5209a service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:41.135677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484cb2b-1814-4c46-9e26-8d1b89e5209a service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:41.135952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484cb2b-1814-4c46-9e26-8d1b89e5209a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:41.150630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484cb2b-1814-4c46-9e26-8d1b89e5209a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:41.152222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484cb2b-1814-4c46-9e26-8d1b89e5209a service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:41.163030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 692/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:43.273868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43c674-ae47-477b-84f7-d3e6a4345fe2 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:43.274373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43c674-ae47-477b-84f7-d3e6a4345fe2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:43.274373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43c674-ae47-477b-84f7-d3e6a4345fe2 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:43.275740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 688/1380] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:41:43.280930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169cf3c-a491-4227-8b23-793b7ee53838 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:43.281322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169cf3c-a491-4227-8b23-793b7ee53838 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:43.281498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9169cf3c-a491-4227-8b23-793b7ee53838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:43.301292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9169cf3c-a491-4227-8b23-793b7ee53838 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:43.302485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9169cf3c-a491-4227-8b23-793b7ee53838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:43.310447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 693/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:43.358679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43c674-ae47-477b-84f7-d3e6a4345fe2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:43.376433 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43c674-ae47-477b-84f7-d3e6a4345fe2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:43.381952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f0a27-7d9c-4143-b14b-1e6f05afa966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:43.382597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f0a27-7d9c-4143-b14b-1e6f05afa966 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:43.382877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f0a27-7d9c-4143-b14b-1e6f05afa966 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:43.383154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f0a27-7d9c-4143-b14b-1e6f05afa966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:43.392214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f0a27-7d9c-4143-b14b-1e6f05afa966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:43.394099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f0a27-7d9c-4143-b14b-1e6f05afa966 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:43.403550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 689/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:45.521872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5563f-3d74-420c-8840-c4e5468c78ec None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:45.522426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5563f-3d74-420c-8840-c4e5468c78ec None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:45.522828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5563f-3d74-420c-8840-c4e5468c78ec None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:45.525299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 694/1383] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:45.532605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32da653-daf0-4673-8511-c1e137a7fd4e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:45.533064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32da653-daf0-4673-8511-c1e137a7fd4e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:45.533226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32da653-daf0-4673-8511-c1e137a7fd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:45.553088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-b32da653-daf0-4673-8511-c1e137a7fd4e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:45.554281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b32da653-daf0-4673-8511-c1e137a7fd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:45.563871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 690/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:45.605310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5563f-3d74-420c-8840-c4e5468c78ec None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:45.620838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5563f-3d74-420c-8840-c4e5468c78ec None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:45.626506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81f841-9d32-49bd-a0bd-3fd940ca426c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:45.627150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81f841-9d32-49bd-a0bd-3fd940ca426c service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:45.627436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81f841-9d32-49bd-a0bd-3fd940ca426c service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:45.627831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81f841-9d32-49bd-a0bd-3fd940ca426c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:45.636732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81f841-9d32-49bd-a0bd-3fd940ca426c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:45.638525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81f841-9d32-49bd-a0bd-3fd940ca426c service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:45.648568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 695/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:47.711726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef7a1f-c6c8-4021-9edf-438eb235ba62 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:47.712133 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef7a1f-c6c8-4021-9edf-438eb235ba62 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:47.712348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef7a1f-c6c8-4021-9edf-438eb235ba62 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:47.714539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 691/1386] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:47.719847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455a17-e246-4559-8604-f6426b8c6ddc None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:47.720256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455a17-e246-4559-8604-f6426b8c6ddc None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:47.720509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b455a17-e246-4559-8604-f6426b8c6ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:47.744979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6b455a17-e246-4559-8604-f6426b8c6ddc None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:47.746759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6b455a17-e246-4559-8604-f6426b8c6ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:47.755164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 696/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:47.800292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef7a1f-c6c8-4021-9edf-438eb235ba62 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:47.816457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef7a1f-c6c8-4021-9edf-438eb235ba62 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:47.822920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67fbb9-e16a-45e7-b629-6daa1af353af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:47.823682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67fbb9-e16a-45e7-b629-6daa1af353af service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:47.824073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67fbb9-e16a-45e7-b629-6daa1af353af service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:47.824430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67fbb9-e16a-45e7-b629-6daa1af353af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:47.833427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67fbb9-e16a-45e7-b629-6daa1af353af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:47.835377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67fbb9-e16a-45e7-b629-6daa1af353af service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:47.847149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 692/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:49.933534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37390e1-6611-4df9-88f9-f38c85d33b23 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:49.934089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37390e1-6611-4df9-88f9-f38c85d33b23 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:49.934089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37390e1-6611-4df9-88f9-f38c85d33b23 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:49.935458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 697/1389] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:49.941913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b39fe-9c1b-4970-af71-095bc20afb0c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:49.942357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b39fe-9c1b-4970-af71-095bc20afb0c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:49.942472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b39fe-9c1b-4970-af71-095bc20afb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:49.963757 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-fd4b39fe-9c1b-4970-af71-095bc20afb0c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:49.965386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-fd4b39fe-9c1b-4970-af71-095bc20afb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:49.975818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 693/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:50.031372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37390e1-6611-4df9-88f9-f38c85d33b23 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:50.046004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37390e1-6611-4df9-88f9-f38c85d33b23 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:50.052342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaadab-d7f4-40b8-a4ee-623a31773bc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:50.053250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaadab-d7f4-40b8-a4ee-623a31773bc9 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:50.053621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaadab-d7f4-40b8-a4ee-623a31773bc9 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:50.053995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaadab-d7f4-40b8-a4ee-623a31773bc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:50.066097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaadab-d7f4-40b8-a4ee-623a31773bc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:50.068163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aaadab-d7f4-40b8-a4ee-623a31773bc9 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:50.080706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 698/1391] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:52.157589 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a69d73-a019-4f8f-84c7-02dce550d2b7 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:52.158115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a69d73-a019-4f8f-84c7-02dce550d2b7 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:52.158535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a69d73-a019-4f8f-84c7-02dce550d2b7 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:52.160354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 694/1392] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:52.165520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232871a1-5dbc-4d16-a7a8-7afac189ceaf None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:52.165924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232871a1-5dbc-4d16-a7a8-7afac189ceaf None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:52.166082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232871a1-5dbc-4d16-a7a8-7afac189ceaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:52.184083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-232871a1-5dbc-4d16-a7a8-7afac189ceaf None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:52.185323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-232871a1-5dbc-4d16-a7a8-7afac189ceaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:52.193846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 699/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:52.238047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a69d73-a019-4f8f-84c7-02dce550d2b7 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:52.252879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a69d73-a019-4f8f-84c7-02dce550d2b7 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:52.258387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc2b02-f704-4fd9-a7d0-e4b5632d5616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:52.259019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc2b02-f704-4fd9-a7d0-e4b5632d5616 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:52.259300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc2b02-f704-4fd9-a7d0-e4b5632d5616 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:52.259576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc2b02-f704-4fd9-a7d0-e4b5632d5616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:52.269183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc2b02-f704-4fd9-a7d0-e4b5632d5616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:52.270767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc2b02-f704-4fd9-a7d0-e4b5632d5616 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:52.280250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 695/1394] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:54.359283 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a0f61-64e9-4ec7-a961-1a0e21ca18bf None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:54.360560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a0f61-64e9-4ec7-a961-1a0e21ca18bf None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:54.360560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a0f61-64e9-4ec7-a961-1a0e21ca18bf None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:54.361999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 700/1395] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:54.367250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c964236-4360-4d96-aa85-567f9cedaec1 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:54.367706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c964236-4360-4d96-aa85-567f9cedaec1 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:54.367961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c964236-4360-4d96-aa85-567f9cedaec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:54.386406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0c964236-4360-4d96-aa85-567f9cedaec1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:54.387610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0c964236-4360-4d96-aa85-567f9cedaec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:54.396784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 696/1396] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:54.441747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a0f61-64e9-4ec7-a961-1a0e21ca18bf None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:54.456095 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a0f61-64e9-4ec7-a961-1a0e21ca18bf None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:54.461486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dc14a-764b-456b-a667-b974ea2d073b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:54.462123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dc14a-764b-456b-a667-b974ea2d073b service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:54.462406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dc14a-764b-456b-a667-b974ea2d073b service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:54.462683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dc14a-764b-456b-a667-b974ea2d073b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:54.470910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dc14a-764b-456b-a667-b974ea2d073b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:54.472489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dc14a-764b-456b-a667-b974ea2d073b service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:54.481950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 701/1397] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:55.376089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276f410-3b7c-42b0-934f-a642bcd33b49 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:55.376668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276f410-3b7c-42b0-934f-a642bcd33b49 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:55.376791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276f410-3b7c-42b0-934f-a642bcd33b49 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:55.378086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 697/1398] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:55.383325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42db7e-13b6-4f5e-a2c4-039e89d4789a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:55.383736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42db7e-13b6-4f5e-a2c4-039e89d4789a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:55.384003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42db7e-13b6-4f5e-a2c4-039e89d4789a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:55.402815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-8a42db7e-13b6-4f5e-a2c4-039e89d4789a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:55.403999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8a42db7e-13b6-4f5e-a2c4-039e89d4789a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:55.411904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 702/1399] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:55.469777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbafd39-2f0c-40ba-a67d-25b0cf37fc93 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:55.470071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbafd39-2f0c-40ba-a67d-25b0cf37fc93 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:55.470354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbafd39-2f0c-40ba-a67d-25b0cf37fc93 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:55.471747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 698/1400] 10.222.0.47 () {54 vars in 962 bytes} [Fri Jun 24 23:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:55.476728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cceccb-38d8-4884-8afa-3c38c52a91e7 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:55.477021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cceccb-38d8-4884-8afa-3c38c52a91e7 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:55.477302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cceccb-38d8-4884-8afa-3c38c52a91e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:55.527291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c7cceccb-38d8-4884-8afa-3c38c52a91e7 None None] MFA Rules not processed for user `71e5ef9dc32f494791ba4addb5151232`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:55.528493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c7cceccb-38d8-4884-8afa-3c38c52a91e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:55.560969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:55.591447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 703/1401] 10.222.0.47 () {58 vars in 1059 bytes} [Fri Jun 24 23:41:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:55.599822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbafd39-2f0c-40ba-a67d-25b0cf37fc93 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:55.610461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbafd39-2f0c-40ba-a67d-25b0cf37fc93 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:55.614015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5c849-36b2-4e39-b1ed-3d14f2c38a74 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71e5ef9dc32f494791ba4addb5151232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:55.614645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5c849-36b2-4e39-b1ed-3d14f2c38a74 service glance] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:55.614926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5c849-36b2-4e39-b1ed-3d14f2c38a74 service glance] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:55.615195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5c849-36b2-4e39-b1ed-3d14f2c38a74 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:55.624089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5c849-36b2-4e39-b1ed-3d14f2c38a74 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:55.626250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5c849-36b2-4e39-b1ed-3d14f2c38a74 service glance] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:55.636257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 699/1402] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:41:56.344669 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b58789-28ab-435b-9996-8bc125e61999 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:56.344960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b58789-28ab-435b-9996-8bc125e61999 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:56.345241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b58789-28ab-435b-9996-8bc125e61999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:56.393242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a7b58789-28ab-435b-9996-8bc125e61999 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:56.394509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a7b58789-28ab-435b-9996-8bc125e61999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:56.460713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 704/1403] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:41:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:56.506512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177e737-614b-4711-9f1d-3b2d884e152f None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:56.506965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177e737-614b-4711-9f1d-3b2d884e152f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:56.507190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177e737-614b-4711-9f1d-3b2d884e152f None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:56.508641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 700/1404] 10.222.0.47 () {54 vars in 942 bytes} [Fri Jun 24 23:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:56.515102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841e365-9636-4e23-b263-c3da7858e2ab None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:56.515425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841e365-9636-4e23-b263-c3da7858e2ab None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:56.515713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841e365-9636-4e23-b263-c3da7858e2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:56.576740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6841e365-9636-4e23-b263-c3da7858e2ab None None] MFA Rules not processed for user `3dd1e02af2374224b968c2644dd0c813`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:56.577992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6841e365-9636-4e23-b263-c3da7858e2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:56.597292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d61848-6456-41bd-a587-f66b7e12e93c None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:56.597582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d61848-6456-41bd-a587-f66b7e12e93c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:56.597864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d61848-6456-41bd-a587-f66b7e12e93c None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:56.599260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 701/1405] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:41:56.604325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de82dc-1a88-48ab-8840-d4a2c0b0e479 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:56.604879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de82dc-1a88-48ab-8840-d4a2c0b0e479 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:56.605164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de82dc-1a88-48ab-8840-d4a2c0b0e479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:56.611768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=61 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:56.624155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-94de82dc-1a88-48ab-8840-d4a2c0b0e479 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:41:56.626046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-94de82dc-1a88-48ab-8840-d4a2c0b0e479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:41:56.633972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 702/1406] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:41:56.660654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 705/1407] 10.222.0.47 () {58 vars in 1039 bytes} [Fri Jun 24 23:41:56 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:41:56.673859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de82dc-1a88-48ab-8840-d4a2c0b0e479 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:56.679668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841e365-9636-4e23-b263-c3da7858e2ab None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:41:56.685342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de82dc-1a88-48ab-8840-d4a2c0b0e479 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:56.691103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293f391-455b-4e53-9fda-cea06d0fc033 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:56.691750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293f391-455b-4e53-9fda-cea06d0fc033 service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:56.692034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293f391-455b-4e53-9fda-cea06d0fc033 service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:56.692315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293f391-455b-4e53-9fda-cea06d0fc033 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:56.692710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841e365-9636-4e23-b263-c3da7858e2ab None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:41:56.698118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc51ce-3d14-445e-b049-6f0d507b3e98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:41:56.698760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc51ce-3d14-445e-b049-6f0d507b3e98 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:41:56.699043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc51ce-3d14-445e-b049-6f0d507b3e98 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:41:56.699322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc51ce-3d14-445e-b049-6f0d507b3e98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:41:56.700872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293f391-455b-4e53-9fda-cea06d0fc033 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:56.702486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293f391-455b-4e53-9fda-cea06d0fc033 service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:56.708415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc51ce-3d14-445e-b049-6f0d507b3e98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:41:56.710133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc51ce-3d14-445e-b049-6f0d507b3e98 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:41:56.710849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 703/1408] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:41:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 23:41:56.715420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:41:56.721515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 706/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:42:06.707367 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=8 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:42:19.490852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf181a-4858-4155-bd66-9a6b37f8da14 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:19.491506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf181a-4858-4155-bd66-9a6b37f8da14 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:19.491506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf181a-4858-4155-bd66-9a6b37f8da14 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:19.492775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 704/1410] 10.222.0.47 () {54 vars in 918 bytes} [Fri Jun 24 23:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:19.497762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22efda1-95ab-48ed-8ca8-e4aee607ddc5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:19.498195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22efda1-95ab-48ed-8ca8-e4aee607ddc5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:19.498325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22efda1-95ab-48ed-8ca8-e4aee607ddc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:19.518081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a22efda1-95ab-48ed-8ca8-e4aee607ddc5 None None] MFA Rules not processed for user `a243d444b7ae487cb0331523d4404d19`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:19.519535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a22efda1-95ab-48ed-8ca8-e4aee607ddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:19.528188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 707/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Fri Jun 24 23:42:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:19.542191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a1c1c-382a-4c1e-bc16-3871a820d034 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:19.542475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a1c1c-382a-4c1e-bc16-3871a820d034 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:19.542751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a1c1c-382a-4c1e-bc16-3871a820d034 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:19.544193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 705/1412] 10.222.0.47 () {54 vars in 918 bytes} [Fri Jun 24 23:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:19.549122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25797fe6-4300-44f8-a7c0-2c3ff1b34f61 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:19.549421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25797fe6-4300-44f8-a7c0-2c3ff1b34f61 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:19.549704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25797fe6-4300-44f8-a7c0-2c3ff1b34f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:19.568239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-25797fe6-4300-44f8-a7c0-2c3ff1b34f61 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:19.568994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-25797fe6-4300-44f8-a7c0-2c3ff1b34f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:19.578063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 708/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Fri Jun 24 23:42:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:22.094828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16fcc6-849f-46b6-8d4d-5a7a791b7da4 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:22.095378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16fcc6-849f-46b6-8d4d-5a7a791b7da4 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:22.095378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16fcc6-849f-46b6-8d4d-5a7a791b7da4 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:22.096756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:22.102044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9308ce-799f-490a-b807-e610d1a164e0 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:22.102461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9308ce-799f-490a-b807-e610d1a164e0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:22.102600 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9308ce-799f-490a-b807-e610d1a164e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:22.120674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7d9308ce-799f-490a-b807-e610d1a164e0 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:22.121876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7d9308ce-799f-490a-b807-e610d1a164e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:22.130218 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:22.173458 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16fcc6-849f-46b6-8d4d-5a7a791b7da4 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:22.188918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16fcc6-849f-46b6-8d4d-5a7a791b7da4 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:22.194279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49bd97-68c2-4e43-bbe9-50bb6d1c3d22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:22.194908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bd97-68c2-4e43-bbe9-50bb6d1c3d22 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:22.195188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bd97-68c2-4e43-bbe9-50bb6d1c3d22 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:22.195465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49bd97-68c2-4e43-bbe9-50bb6d1c3d22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:22.203410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49bd97-68c2-4e43-bbe9-50bb6d1c3d22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:22.205159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49bd97-68c2-4e43-bbe9-50bb6d1c3d22 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:22.214492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 707/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:23.133315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c5eb8-2353-457a-81f1-0974cb5f321a None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:23.133876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c5eb8-2353-457a-81f1-0974cb5f321a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:23.133876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c5eb8-2353-457a-81f1-0974cb5f321a None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:23.135246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:23.140630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8264b8-ad3a-4b03-afa7-5c190df80a3b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:23.141637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8264b8-ad3a-4b03-afa7-5c190df80a3b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:23.141762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8264b8-ad3a-4b03-afa7-5c190df80a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:23.160820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-de8264b8-ad3a-4b03-afa7-5c190df80a3b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:23.162055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-de8264b8-ad3a-4b03-afa7-5c190df80a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:23.169838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:23.319948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c5eb8-2353-457a-81f1-0974cb5f321a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:23.334503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c5eb8-2353-457a-81f1-0974cb5f321a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:23.339787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d190f90-d92e-43c8-bfa2-2c7108bb76ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:23.340400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d190f90-d92e-43c8-bfa2-2c7108bb76ec service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:23.340719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d190f90-d92e-43c8-bfa2-2c7108bb76ec service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:23.341001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d190f90-d92e-43c8-bfa2-2c7108bb76ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:23.349509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d190f90-d92e-43c8-bfa2-2c7108bb76ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:23.351071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d190f90-d92e-43c8-bfa2-2c7108bb76ec service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:23.361217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 711/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:24.244497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe39ed-47eb-4a50-b0de-2a38948366bb None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:24.245049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe39ed-47eb-4a50-b0de-2a38948366bb None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:24.245049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe39ed-47eb-4a50-b0de-2a38948366bb None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:24.246357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:24.251953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d880e2-4cc8-44f2-9e0a-854d51573e2a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:24.252382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d880e2-4cc8-44f2-9e0a-854d51573e2a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:24.252510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d880e2-4cc8-44f2-9e0a-854d51573e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:24.270609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-15d880e2-4cc8-44f2-9e0a-854d51573e2a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:24.271803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-15d880e2-4cc8-44f2-9e0a-854d51573e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:24.279515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:24.433147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe39ed-47eb-4a50-b0de-2a38948366bb None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:24.447807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe39ed-47eb-4a50-b0de-2a38948366bb None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:24.453123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f52fd3-05ac-4713-80fd-cd58cbcd5e6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:24.453776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f52fd3-05ac-4713-80fd-cd58cbcd5e6c service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:24.454056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f52fd3-05ac-4713-80fd-cd58cbcd5e6c service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:24.454334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f52fd3-05ac-4713-80fd-cd58cbcd5e6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:24.462320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f52fd3-05ac-4713-80fd-cd58cbcd5e6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:24.464147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f52fd3-05ac-4713-80fd-cd58cbcd5e6c service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:24.473815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 710/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:25.350067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d7bac-cb67-4ba6-bc76-9513de2a28b0 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:25.350900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d7bac-cb67-4ba6-bc76-9513de2a28b0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:25.350900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d7bac-cb67-4ba6-bc76-9513de2a28b0 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:25.352006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:25.357122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fb495-2028-4fc3-9bfc-c5156d1c7de1 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:25.357548 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fb495-2028-4fc3-9bfc-c5156d1c7de1 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:25.357676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871fb495-2028-4fc3-9bfc-c5156d1c7de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:25.378942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-871fb495-2028-4fc3-9bfc-c5156d1c7de1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:25.380157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-871fb495-2028-4fc3-9bfc-c5156d1c7de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:25.388016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:25.542353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d7bac-cb67-4ba6-bc76-9513de2a28b0 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:25.556559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d7bac-cb67-4ba6-bc76-9513de2a28b0 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:25.561873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b6743-3021-440c-ae9c-c241c350ffb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:25.562504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b6743-3021-440c-ae9c-c241c350ffb8 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:25.563274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b6743-3021-440c-ae9c-c241c350ffb8 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:25.563587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b6743-3021-440c-ae9c-c241c350ffb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:25.571608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b6743-3021-440c-ae9c-c241c350ffb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:25.573224 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b6743-3021-440c-ae9c-c241c350ffb8 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:25.583488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 714/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:26.462693 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d839b-a27a-46b4-8a75-f8d28d6dc208 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:26.463234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d839b-a27a-46b4-8a75-f8d28d6dc208 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:26.463356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d839b-a27a-46b4-8a75-f8d28d6dc208 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:26.464632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:26.469564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf2e63-65a8-44c3-9dd7-9d7552386eff None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:26.469990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf2e63-65a8-44c3-9dd7-9d7552386eff None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:26.470115 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf2e63-65a8-44c3-9dd7-9d7552386eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:26.488210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7bbf2e63-65a8-44c3-9dd7-9d7552386eff None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:26.489432 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7bbf2e63-65a8-44c3-9dd7-9d7552386eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:26.497068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:26.648728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d839b-a27a-46b4-8a75-f8d28d6dc208 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:26.663434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d839b-a27a-46b4-8a75-f8d28d6dc208 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:26.668746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5e582-22ac-4fbd-b18c-d370872a411e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:26.669378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5e582-22ac-4fbd-b18c-d370872a411e service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:26.669662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5e582-22ac-4fbd-b18c-d370872a411e service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:26.669940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5e582-22ac-4fbd-b18c-d370872a411e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:26.678128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5e582-22ac-4fbd-b18c-d370872a411e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:26.679686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5e582-22ac-4fbd-b18c-d370872a411e service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:26.691257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 713/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:27.505497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a08f27-e4f4-4616-b215-87deaa17381f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:27.506043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a08f27-e4f4-4616-b215-87deaa17381f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:27.506043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a08f27-e4f4-4616-b215-87deaa17381f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:27.507392 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:42:27.512346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f5460-d758-4373-bfae-985a8b731982 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:27.512749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f5460-d758-4373-bfae-985a8b731982 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:27.512929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f5460-d758-4373-bfae-985a8b731982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:27.532101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-cd0f5460-d758-4373-bfae-985a8b731982 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:27.533321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-cd0f5460-d758-4373-bfae-985a8b731982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:27.541146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:27.692356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a08f27-e4f4-4616-b215-87deaa17381f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:27.708469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a08f27-e4f4-4616-b215-87deaa17381f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:27.719501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aafcb6-4632-46f5-98f1-42540c6841ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:27.720152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aafcb6-4632-46f5-98f1-42540c6841ec service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:27.720438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aafcb6-4632-46f5-98f1-42540c6841ec service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:27.720749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aafcb6-4632-46f5-98f1-42540c6841ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:27.729286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aafcb6-4632-46f5-98f1-42540c6841ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:27.730878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aafcb6-4632-46f5-98f1-42540c6841ec service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:27.740812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 717/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:28.570797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fa65b-b03b-414e-9228-bbcc2bc89b96 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:28.571371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fa65b-b03b-414e-9228-bbcc2bc89b96 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:28.571371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fa65b-b03b-414e-9228-bbcc2bc89b96 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:28.572792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:28.578104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ea195-1e01-4fea-a3aa-8b8fd5196184 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:28.579146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ea195-1e01-4fea-a3aa-8b8fd5196184 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:28.579405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ea195-1e01-4fea-a3aa-8b8fd5196184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:28.597110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2e3ea195-1e01-4fea-a3aa-8b8fd5196184 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:28.598374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2e3ea195-1e01-4fea-a3aa-8b8fd5196184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:28.606218 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:42:28.760658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fa65b-b03b-414e-9228-bbcc2bc89b96 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:28.775916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fa65b-b03b-414e-9228-bbcc2bc89b96 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:28.781247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe03f8-b9ba-4bcd-9b03-21be429e8d38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:28.782016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe03f8-b9ba-4bcd-9b03-21be429e8d38 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:28.782295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe03f8-b9ba-4bcd-9b03-21be429e8d38 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:28.782571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe03f8-b9ba-4bcd-9b03-21be429e8d38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:28.791406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe03f8-b9ba-4bcd-9b03-21be429e8d38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:28.793166 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe03f8-b9ba-4bcd-9b03-21be429e8d38 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:28.803434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 716/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:29.635559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5e511-88ff-47e1-9383-963b5559e57a None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:29.636142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5e511-88ff-47e1-9383-963b5559e57a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:29.636142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5e511-88ff-47e1-9383-963b5559e57a None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:29.637530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:29.643128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac2d0a-9c18-4b40-93be-d1d320234a4d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:29.643578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac2d0a-9c18-4b40-93be-d1d320234a4d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:29.643955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac2d0a-9c18-4b40-93be-d1d320234a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:29.663069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-07ac2d0a-9c18-4b40-93be-d1d320234a4d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:29.664279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-07ac2d0a-9c18-4b40-93be-d1d320234a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:29.672297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:29.823586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5e511-88ff-47e1-9383-963b5559e57a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:29.838326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5e511-88ff-47e1-9383-963b5559e57a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:29.843597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542f521-7291-443c-998d-e10a15a043ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:29.845089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542f521-7291-443c-998d-e10a15a043ed service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:29.845368 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542f521-7291-443c-998d-e10a15a043ed service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:29.845641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542f521-7291-443c-998d-e10a15a043ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:29.853556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542f521-7291-443c-998d-e10a15a043ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:29.855118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542f521-7291-443c-998d-e10a15a043ed service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:29.864342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 720/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:30.667453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6735a0d-221a-4e12-b784-4f2af57cbac2 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:30.668006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6735a0d-221a-4e12-b784-4f2af57cbac2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:30.668006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6735a0d-221a-4e12-b784-4f2af57cbac2 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:30.669401 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:30.674431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498e88b-9fad-4efd-862d-8f9b69c931bf None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:30.674849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498e88b-9fad-4efd-862d-8f9b69c931bf None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:30.675046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498e88b-9fad-4efd-862d-8f9b69c931bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:30.693212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-8498e88b-9fad-4efd-862d-8f9b69c931bf None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:30.694396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8498e88b-9fad-4efd-862d-8f9b69c931bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:30.702025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:30.852018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6735a0d-221a-4e12-b784-4f2af57cbac2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:30.866516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6735a0d-221a-4e12-b784-4f2af57cbac2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:30.871858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98e3bb-d56c-4677-a75e-79e3dee8d7ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:30.872489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98e3bb-d56c-4677-a75e-79e3dee8d7ef service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:30.872808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98e3bb-d56c-4677-a75e-79e3dee8d7ef service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:30.873092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98e3bb-d56c-4677-a75e-79e3dee8d7ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:30.881741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98e3bb-d56c-4677-a75e-79e3dee8d7ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:30.883313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98e3bb-d56c-4677-a75e-79e3dee8d7ef service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:30.893424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 719/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:32.868963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d486b3-8f11-46ec-9854-aa543fb29b41 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:32.869516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d486b3-8f11-46ec-9854-aa543fb29b41 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:32.869623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d486b3-8f11-46ec-9854-aa543fb29b41 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:32.870929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:32.876514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052225d-f498-4a1c-b073-a2bc88bd8b61 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:32.876999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052225d-f498-4a1c-b073-a2bc88bd8b61 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:32.877103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052225d-f498-4a1c-b073-a2bc88bd8b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:32.896040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-5052225d-f498-4a1c-b073-a2bc88bd8b61 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:32.897274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5052225d-f498-4a1c-b073-a2bc88bd8b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:32.905117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:32.943998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d486b3-8f11-46ec-9854-aa543fb29b41 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:32.955318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d486b3-8f11-46ec-9854-aa543fb29b41 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:32.958881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9b40b-37a9-42d2-a7fa-5e1a07d4e922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:32.959520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9b40b-37a9-42d2-a7fa-5e1a07d4e922 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:32.959803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9b40b-37a9-42d2-a7fa-5e1a07d4e922 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:32.960083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9b40b-37a9-42d2-a7fa-5e1a07d4e922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:32.969401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9b40b-37a9-42d2-a7fa-5e1a07d4e922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:32.970852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9b40b-37a9-42d2-a7fa-5e1a07d4e922 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:32.980765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 723/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:35.159281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da3433-8fdb-4cf4-9a60-3b177817986a None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:35.159849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da3433-8fdb-4cf4-9a60-3b177817986a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:35.159849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da3433-8fdb-4cf4-9a60-3b177817986a None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:35.161479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:35.166981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280ff75-0c46-4593-b202-df71d3b7ef92 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:35.167420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280ff75-0c46-4593-b202-df71d3b7ef92 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:35.167524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280ff75-0c46-4593-b202-df71d3b7ef92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:35.185801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1280ff75-0c46-4593-b202-df71d3b7ef92 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:35.186997 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1280ff75-0c46-4593-b202-df71d3b7ef92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:35.195354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:42:35.233067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da3433-8fdb-4cf4-9a60-3b177817986a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:35.244332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da3433-8fdb-4cf4-9a60-3b177817986a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:35.247900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7dc22-951f-4456-a92c-0a27f4f41560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:35.248531 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7dc22-951f-4456-a92c-0a27f4f41560 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:35.248844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7dc22-951f-4456-a92c-0a27f4f41560 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:35.249123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7dc22-951f-4456-a92c-0a27f4f41560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:35.258049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7dc22-951f-4456-a92c-0a27f4f41560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:35.259468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7dc22-951f-4456-a92c-0a27f4f41560 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:35.269305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 722/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:37.709817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e3f57-36a8-4c51-b827-3fb346a4f05c None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:37.710373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e3f57-36a8-4c51-b827-3fb346a4f05c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:37.710373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e3f57-36a8-4c51-b827-3fb346a4f05c None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:37.711750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:37.716972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6a45d-fb61-4868-b91e-521c8a768236 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:37.717395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6a45d-fb61-4868-b91e-521c8a768236 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:37.717553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6a45d-fb61-4868-b91e-521c8a768236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:37.735586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-96d6a45d-fb61-4868-b91e-521c8a768236 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:37.736797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-96d6a45d-fb61-4868-b91e-521c8a768236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:37.744511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:37.784842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e3f57-36a8-4c51-b827-3fb346a4f05c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:37.795302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e3f57-36a8-4c51-b827-3fb346a4f05c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:37.799067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19279db-65e3-4fde-b19b-ecfecc912c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:37.799701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19279db-65e3-4fde-b19b-ecfecc912c6a service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:37.799981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19279db-65e3-4fde-b19b-ecfecc912c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:37.800270 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19279db-65e3-4fde-b19b-ecfecc912c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:37.809016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19279db-65e3-4fde-b19b-ecfecc912c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:37.810443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19279db-65e3-4fde-b19b-ecfecc912c6a service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:37.820084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 726/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:40.011694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3273b-bf35-4ec7-a2e8-21d20c8e5652 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:40.012227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3273b-bf35-4ec7-a2e8-21d20c8e5652 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:40.012227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3273b-bf35-4ec7-a2e8-21d20c8e5652 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:40.013669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:40.020718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6972664-e385-4f96-96d7-ff82cbd7c3c5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:40.021188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6972664-e385-4f96-96d7-ff82cbd7c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:40.021299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6972664-e385-4f96-96d7-ff82cbd7c3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:40.040506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-e6972664-e385-4f96-96d7-ff82cbd7c3c5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:40.041752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e6972664-e385-4f96-96d7-ff82cbd7c3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:40.050136 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:40.089711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3273b-bf35-4ec7-a2e8-21d20c8e5652 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:40.101075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3273b-bf35-4ec7-a2e8-21d20c8e5652 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:40.104563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb81a8b-5314-4d08-8f64-6e6d3599cb70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:40.105242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb81a8b-5314-4d08-8f64-6e6d3599cb70 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:40.105526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb81a8b-5314-4d08-8f64-6e6d3599cb70 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:40.105808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb81a8b-5314-4d08-8f64-6e6d3599cb70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:40.114771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb81a8b-5314-4d08-8f64-6e6d3599cb70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:40.116198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb81a8b-5314-4d08-8f64-6e6d3599cb70 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:40.126028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 725/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:42.317854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d2dec-ca5a-4c76-b1dd-dc42216152bb None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:42.318386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d2dec-ca5a-4c76-b1dd-dc42216152bb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:42.318386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d2dec-ca5a-4c76-b1dd-dc42216152bb None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:42.319699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:42.324991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f3bbe-7759-4f87-8b76-b0f95c8bce82 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:42.325378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f3bbe-7759-4f87-8b76-b0f95c8bce82 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:42.325547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f3bbe-7759-4f87-8b76-b0f95c8bce82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:42.344896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-aa1f3bbe-7759-4f87-8b76-b0f95c8bce82 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:42.346141 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-aa1f3bbe-7759-4f87-8b76-b0f95c8bce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:42.355707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:42.397209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d2dec-ca5a-4c76-b1dd-dc42216152bb None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:42.408887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d2dec-ca5a-4c76-b1dd-dc42216152bb None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:42.412378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469879ed-7df1-435f-a706-4ca200b79f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:42.413057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469879ed-7df1-435f-a706-4ca200b79f5f service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:42.413342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469879ed-7df1-435f-a706-4ca200b79f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:42.413617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469879ed-7df1-435f-a706-4ca200b79f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:42.422652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469879ed-7df1-435f-a706-4ca200b79f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:42.424096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469879ed-7df1-435f-a706-4ca200b79f5f service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:42.433395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 729/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:44.519603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22afe00-45c1-49a6-9721-e2a976aa19f2 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:44.520174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22afe00-45c1-49a6-9721-e2a976aa19f2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:44.520174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22afe00-45c1-49a6-9721-e2a976aa19f2 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:44.521563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:42:44.529775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b6976-0c9b-4371-831f-d2d781e033e5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:44.530203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b6976-0c9b-4371-831f-d2d781e033e5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:44.530343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b6976-0c9b-4371-831f-d2d781e033e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:44.548315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-314b6976-0c9b-4371-831f-d2d781e033e5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:44.549553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-314b6976-0c9b-4371-831f-d2d781e033e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:44.557262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:44.596321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22afe00-45c1-49a6-9721-e2a976aa19f2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:44.606803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22afe00-45c1-49a6-9721-e2a976aa19f2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:44.610352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2d84e-bf26-41c2-a27a-71726db9fdbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:44.610987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d84e-bf26-41c2-a27a-71726db9fdbb service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:44.611267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d84e-bf26-41c2-a27a-71726db9fdbb service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:44.611539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d84e-bf26-41c2-a27a-71726db9fdbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:44.620219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2d84e-bf26-41c2-a27a-71726db9fdbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:44.621670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2d84e-bf26-41c2-a27a-71726db9fdbb service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:44.631140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 728/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:46.731591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c29cdd-db39-4b97-8a60-1e5b10660cc5 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:46.732158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c29cdd-db39-4b97-8a60-1e5b10660cc5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:46.732158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c29cdd-db39-4b97-8a60-1e5b10660cc5 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:46.733544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:46.739226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45f898-4f76-43b0-89cf-fb7997fe0b8f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:46.739654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45f898-4f76-43b0-89cf-fb7997fe0b8f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:46.739799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45f898-4f76-43b0-89cf-fb7997fe0b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:46.745845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=70 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:42:46.758981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-ea45f898-4f76-43b0-89cf-fb7997fe0b8f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:46.760200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ea45f898-4f76-43b0-89cf-fb7997fe0b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:46.768019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:46.805808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c29cdd-db39-4b97-8a60-1e5b10660cc5 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:46.811059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=73 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:42:46.818196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c29cdd-db39-4b97-8a60-1e5b10660cc5 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:46.821643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709de1f8-06c2-4a71-991d-20ffd28c8ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:46.822274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709de1f8-06c2-4a71-991d-20ffd28c8ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:46.822557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709de1f8-06c2-4a71-991d-20ffd28c8ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:46.822833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709de1f8-06c2-4a71-991d-20ffd28c8ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:46.831862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709de1f8-06c2-4a71-991d-20ffd28c8ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:46.833327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709de1f8-06c2-4a71-991d-20ffd28c8ee0 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:46.842615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 732/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:48.943159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf7019-b600-4167-8125-7eead7a60b97 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:48.943698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf7019-b600-4167-8125-7eead7a60b97 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:48.943698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf7019-b600-4167-8125-7eead7a60b97 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:48.945139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:48.950313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a423005-77a1-466c-9ee3-906736d2f32c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:48.950762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a423005-77a1-466c-9ee3-906736d2f32c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:48.950853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a423005-77a1-466c-9ee3-906736d2f32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:48.969725 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1a423005-77a1-466c-9ee3-906736d2f32c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:48.970937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1a423005-77a1-466c-9ee3-906736d2f32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:48.978818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:49.017393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf7019-b600-4167-8125-7eead7a60b97 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:49.028251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf7019-b600-4167-8125-7eead7a60b97 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:49.032533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecc47c-f877-452c-a36f-fc1023547344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:49.033252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc47c-f877-452c-a36f-fc1023547344 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:49.033534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc47c-f877-452c-a36f-fc1023547344 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:49.033836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecc47c-f877-452c-a36f-fc1023547344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:49.042948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc47c-f877-452c-a36f-fc1023547344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:49.044379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecc47c-f877-452c-a36f-fc1023547344 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:49.055152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 731/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:51.158721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e97735-ab34-456e-9ee8-791ad0e0f280 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:51.159282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e97735-ab34-456e-9ee8-791ad0e0f280 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:51.159282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e97735-ab34-456e-9ee8-791ad0e0f280 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:51.160646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:51.165961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656a832-2f1d-4d26-9269-926037be21a8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:51.166371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656a832-2f1d-4d26-9269-926037be21a8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:51.166517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656a832-2f1d-4d26-9269-926037be21a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:51.184397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8656a832-2f1d-4d26-9269-926037be21a8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:51.185624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8656a832-2f1d-4d26-9269-926037be21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:51.194592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:42:51.231165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e97735-ab34-456e-9ee8-791ad0e0f280 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:51.242448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e97735-ab34-456e-9ee8-791ad0e0f280 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:51.245908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de945227-6b70-4419-971d-316da9a1637d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:51.246539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de945227-6b70-4419-971d-316da9a1637d service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:51.246818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de945227-6b70-4419-971d-316da9a1637d service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:51.247087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de945227-6b70-4419-971d-316da9a1637d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:51.256513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de945227-6b70-4419-971d-316da9a1637d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:51.257955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de945227-6b70-4419-971d-316da9a1637d service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:51.267852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 735/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:42:53.330618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2fda0-a65a-4174-a249-7d45997ba6b9 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:53.331128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2fda0-a65a-4174-a249-7d45997ba6b9 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:53.331128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2fda0-a65a-4174-a249-7d45997ba6b9 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:53.332514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:42:53.337659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1d546-ea84-4738-85ed-09a540cf932f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:53.338072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1d546-ea84-4738-85ed-09a540cf932f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:53.338209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1d546-ea84-4738-85ed-09a540cf932f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:53.355345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-60a1d546-ea84-4738-85ed-09a540cf932f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:42:53.356530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-60a1d546-ea84-4738-85ed-09a540cf932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:42:53.364193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:42:53.400355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2fda0-a65a-4174-a249-7d45997ba6b9 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:42:53.411234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2fda0-a65a-4174-a249-7d45997ba6b9 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:42:53.414731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4336082-2411-49c6-bfa3-1bf3c235ad46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:42:53.415352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4336082-2411-49c6-bfa3-1bf3c235ad46 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:42:53.415633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4336082-2411-49c6-bfa3-1bf3c235ad46 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:42:53.415908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4336082-2411-49c6-bfa3-1bf3c235ad46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:42:53.424756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4336082-2411-49c6-bfa3-1bf3c235ad46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:42:53.426182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4336082-2411-49c6-bfa3-1bf3c235ad46 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:42:53.435448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 734/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:42:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:43:03.350347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=17 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:43:03.431341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=17 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:43:13.414842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14003f73-7acd-4093-9b82-5384c3bd0854 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:13.415416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14003f73-7acd-4093-9b82-5384c3bd0854 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:13.415416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14003f73-7acd-4093-9b82-5384c3bd0854 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:13.416876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 737/1471] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:43:13.422908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51981fdd-da63-435e-b206-7da05dd91d80 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:13.423413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51981fdd-da63-435e-b206-7da05dd91d80 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:13.423521 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51981fdd-da63-435e-b206-7da05dd91d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:13.440913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-51981fdd-da63-435e-b206-7da05dd91d80 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:43:13.442115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-51981fdd-da63-435e-b206-7da05dd91d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:43:13.450126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 735/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:43:13.497519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfbca6-a85a-4294-b006-2c0248b190f5 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:13.497810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfbca6-a85a-4294-b006-2c0248b190f5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:13.498091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfbca6-a85a-4294-b006-2c0248b190f5 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:13.499436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 738/1473] 10.222.0.47 () {54 vars in 962 bytes} [Fri Jun 24 23:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:43:13.504605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baeda07-f663-4fc3-9580-a27f0083d24e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:13.504907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baeda07-f663-4fc3-9580-a27f0083d24e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:13.505191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baeda07-f663-4fc3-9580-a27f0083d24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:13.525063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-9baeda07-f663-4fc3-9580-a27f0083d24e None None] MFA Rules not processed for user `71e5ef9dc32f494791ba4addb5151232`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:43:13.526254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9baeda07-f663-4fc3-9580-a27f0083d24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:43:13.533193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 736/1474] 10.222.0.47 () {58 vars in 1059 bytes} [Fri Jun 24 23:43:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:43:13.540494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfbca6-a85a-4294-b006-2c0248b190f5 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:43:13.552313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfbca6-a85a-4294-b006-2c0248b190f5 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:43:13.555897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536cee9-0a09-4aa9-81a9-6e9abc57beff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71e5ef9dc32f494791ba4addb5151232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:43:13.556562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536cee9-0a09-4aa9-81a9-6e9abc57beff service glance] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:13.556858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536cee9-0a09-4aa9-81a9-6e9abc57beff service glance] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:13.557138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536cee9-0a09-4aa9-81a9-6e9abc57beff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:13.567567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536cee9-0a09-4aa9-81a9-6e9abc57beff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:43:13.569105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0536cee9-0a09-4aa9-81a9-6e9abc57beff service glance] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:43:13.580779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 739/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:43:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:43:15.717595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3f4b3-16b4-4e7d-af34-8084d41e26f0 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:15.718175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3f4b3-16b4-4e7d-af34-8084d41e26f0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:15.718175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3f4b3-16b4-4e7d-af34-8084d41e26f0 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:15.720051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 737/1476] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:43:15.725358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ebf15-2c7b-48c5-8fcb-8d5eafe942f1 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:15.725793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ebf15-2c7b-48c5-8fcb-8d5eafe942f1 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:15.725955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ebf15-2c7b-48c5-8fcb-8d5eafe942f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:15.744695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-442ebf15-2c7b-48c5-8fcb-8d5eafe942f1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:43:15.745925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-442ebf15-2c7b-48c5-8fcb-8d5eafe942f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:43:15.753799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 740/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:43:15.777263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3f4b3-16b4-4e7d-af34-8084d41e26f0 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:43:15.790565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3f4b3-16b4-4e7d-af34-8084d41e26f0 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:43:15.795068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7e4f2-f784-43d7-9006-b3a0df3a1ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:43:15.795716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7e4f2-f784-43d7-9006-b3a0df3a1ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:15.796003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7e4f2-f784-43d7-9006-b3a0df3a1ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:15.796309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7e4f2-f784-43d7-9006-b3a0df3a1ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:15.797655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7e4f2-f784-43d7-9006-b3a0df3a1ff8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:43:15.799430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WARNING py.warnings [None req-ceb7e4f2-f784-43d7-9006-b3a0df3a1ff8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:43:15.799430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: warnings.warn(msg) Jun 24 23:43:15.799430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: Jun 24 23:43:15.800563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7e4f2-f784-43d7-9006-b3a0df3a1ff8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:43:15.819364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 738/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Fri Jun 24 23:43:15 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:43:17.874091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae851c-99c8-4d39-a66f-4a5b24aa49e9 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:17.874671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae851c-99c8-4d39-a66f-4a5b24aa49e9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:17.874671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae851c-99c8-4d39-a66f-4a5b24aa49e9 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:17.876945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 741/1479] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:43:17.882191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1ff73-75cc-49a9-9950-8d1c0b2ad4e5 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:17.882588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1ff73-75cc-49a9-9950-8d1c0b2ad4e5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:17.882751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1ff73-75cc-49a9-9950-8d1c0b2ad4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:17.901884 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-b6f1ff73-75cc-49a9-9950-8d1c0b2ad4e5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:43:17.903062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b6f1ff73-75cc-49a9-9950-8d1c0b2ad4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:43:17.910963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 739/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:43:17.954777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae851c-99c8-4d39-a66f-4a5b24aa49e9 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:43:17.969512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae851c-99c8-4d39-a66f-4a5b24aa49e9 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:43:17.974786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22fdfe-99df-40af-9c8b-5b0acbd50ed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:43:17.975418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22fdfe-99df-40af-9c8b-5b0acbd50ed5 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:17.975696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22fdfe-99df-40af-9c8b-5b0acbd50ed5 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:17.975964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22fdfe-99df-40af-9c8b-5b0acbd50ed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:17.983949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22fdfe-99df-40af-9c8b-5b0acbd50ed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:43:17.985644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22fdfe-99df-40af-9c8b-5b0acbd50ed5 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:43:17.995070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 742/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:43:20.246048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca3716-4aa8-49ed-a3c6-cb475f87b2a2 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:20.246555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca3716-4aa8-49ed-a3c6-cb475f87b2a2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:20.246555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca3716-4aa8-49ed-a3c6-cb475f87b2a2 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:20.247890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 740/1482] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:43:20.254101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a0893-b3e4-4542-934b-309bddb3f3bc None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:20.254534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a0893-b3e4-4542-934b-309bddb3f3bc None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:20.254662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a0893-b3e4-4542-934b-309bddb3f3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:20.273326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a22a0893-b3e4-4542-934b-309bddb3f3bc None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:43:20.274526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a22a0893-b3e4-4542-934b-309bddb3f3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:43:20.282718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 743/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:43:20.319427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca3716-4aa8-49ed-a3c6-cb475f87b2a2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:43:20.330670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca3716-4aa8-49ed-a3c6-cb475f87b2a2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:43:20.334191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acca77-0e48-4e4c-8213-449acd445cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:43:20.334799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acca77-0e48-4e4c-8213-449acd445cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:20.335080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acca77-0e48-4e4c-8213-449acd445cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:20.335351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acca77-0e48-4e4c-8213-449acd445cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:20.344139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acca77-0e48-4e4c-8213-449acd445cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:43:20.345586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acca77-0e48-4e4c-8213-449acd445cd0 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:43:20.354819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 741/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:43:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:43:22.460229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7336fcc-bcbd-489a-b936-7e4580d23aa7 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:22.460884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7336fcc-bcbd-489a-b936-7e4580d23aa7 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:22.460884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7336fcc-bcbd-489a-b936-7e4580d23aa7 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:22.462174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 744/1485] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:43:22.467286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d6072-0400-471b-9252-f08b5007a070 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:22.467683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d6072-0400-471b-9252-f08b5007a070 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:22.467843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d6072-0400-471b-9252-f08b5007a070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:22.486226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-666d6072-0400-471b-9252-f08b5007a070 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:43:22.487418 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-666d6072-0400-471b-9252-f08b5007a070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:43:22.495209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 742/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:43:22.533704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7336fcc-bcbd-489a-b936-7e4580d23aa7 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:43:22.544461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7336fcc-bcbd-489a-b936-7e4580d23aa7 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:43:22.548346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bb5b1-7e26-44e1-9149-c7c2360ba6c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:43:22.549018 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bb5b1-7e26-44e1-9149-c7c2360ba6c1 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:22.549332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bb5b1-7e26-44e1-9149-c7c2360ba6c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:22.549613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bb5b1-7e26-44e1-9149-c7c2360ba6c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:22.558403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bb5b1-7e26-44e1-9149-c7c2360ba6c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:43:22.559832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bb5b1-7e26-44e1-9149-c7c2360ba6c1 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:43:22.570873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 745/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:43:24.751861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28757c-7ef6-456c-becd-0aa481c494fc None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:24.752434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28757c-7ef6-456c-becd-0aa481c494fc None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:24.752434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28757c-7ef6-456c-becd-0aa481c494fc None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:24.753806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 743/1488] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:43:24.759261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c227f6d-967d-4e3c-9108-c7ebe71ea57b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:24.759684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c227f6d-967d-4e3c-9108-c7ebe71ea57b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:24.759804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c227f6d-967d-4e3c-9108-c7ebe71ea57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:24.777752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3c227f6d-967d-4e3c-9108-c7ebe71ea57b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:43:24.778938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3c227f6d-967d-4e3c-9108-c7ebe71ea57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:43:24.787587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 746/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:43:24.827040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28757c-7ef6-456c-becd-0aa481c494fc None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:43:24.837829 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28757c-7ef6-456c-becd-0aa481c494fc None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:43:24.841363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a24ef-2b65-4e1c-9a0b-3e3226a56d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:43:24.841994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a24ef-2b65-4e1c-9a0b-3e3226a56d95 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:24.842281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a24ef-2b65-4e1c-9a0b-3e3226a56d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:24.842561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a24ef-2b65-4e1c-9a0b-3e3226a56d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:24.852055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a24ef-2b65-4e1c-9a0b-3e3226a56d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:43:24.853508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a24ef-2b65-4e1c-9a0b-3e3226a56d95 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:43:24.863559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 744/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:43:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:43:27.103692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e9e82-2190-40b2-a33c-d171be313e4b None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:27.104273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e9e82-2190-40b2-a33c-d171be313e4b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:27.104372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e9e82-2190-40b2-a33c-d171be313e4b None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:27.105835 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 747/1491] 10.222.0.47 () {54 vars in 926 bytes} [Fri Jun 24 23:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:43:27.111663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481db0-e9e3-45c9-b433-2585210e2bc6 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:27.112103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481db0-e9e3-45c9-b433-2585210e2bc6 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:27.112211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19481db0-e9e3-45c9-b433-2585210e2bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:27.129438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-19481db0-e9e3-45c9-b433-2585210e2bc6 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:43:27.130611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-19481db0-e9e3-45c9-b433-2585210e2bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:43:27.138701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 745/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Fri Jun 24 23:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:43:27.178770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e9e82-2190-40b2-a33c-d171be313e4b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:43:27.189774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e9e82-2190-40b2-a33c-d171be313e4b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:43:27.193381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd263c6-789f-4bb5-9735-7a38db45467a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:43:27.194021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd263c6-789f-4bb5-9735-7a38db45467a service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:43:27.194306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd263c6-789f-4bb5-9735-7a38db45467a service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:43:27.194587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd263c6-789f-4bb5-9735-7a38db45467a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:43:27.204284 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd263c6-789f-4bb5-9735-7a38db45467a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:43:27.205748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd263c6-789f-4bb5-9735-7a38db45467a service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:43:27.215216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 748/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:43:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:43:37.124382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=33 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:43:37.210977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=33 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:44:53.406735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c32fe-f9d9-4c6b-86c7-36598e5fc6c6 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.407328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c32fe-f9d9-4c6b-86c7-36598e5fc6c6 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.407328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c32fe-f9d9-4c6b-86c7-36598e5fc6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.426246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-3c1c32fe-f9d9-4c6b-86c7-36598e5fc6c6 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:44:53.427445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3c1c32fe-f9d9-4c6b-86c7-36598e5fc6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:44:53.435175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 746/1494] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:44:53.443067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a43202-82f3-4c14-96cb-2ac03a557918 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.443600 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a43202-82f3-4c14-96cb-2ac03a557918 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.443707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a43202-82f3-4c14-96cb-2ac03a557918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.462354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f7a43202-82f3-4c14-96cb-2ac03a557918 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:44:53.463539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f7a43202-82f3-4c14-96cb-2ac03a557918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:44:53.472315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 749/1495] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:44:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:44:53.476851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c32fe-f9d9-4c6b-86c7-36598e5fc6c6 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:53.486667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c32fe-f9d9-4c6b-86c7-36598e5fc6c6 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:53.491154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:53.491805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.492091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.492373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.493146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:44:53.493462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:53.494475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WARNING py.warnings [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:44:53.494475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: warnings.warn(msg) Jun 24 23:44:53.494475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: Jun 24 23:44:53.495351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:53.507215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 747/1496] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:44:53.515635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a43202-82f3-4c14-96cb-2ac03a557918 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:53.526552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a43202-82f3-4c14-96cb-2ac03a557918 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:53.531500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fcd5b-443a-4364-9872-5d9c84cabc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:53.532170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fcd5b-443a-4364-9872-5d9c84cabc11 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.532449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fcd5b-443a-4364-9872-5d9c84cabc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.532758 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50fcd5b-443a-4364-9872-5d9c84cabc11 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.533442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fcd5b-443a-4364-9872-5d9c84cabc11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:53.535221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WARNING py.warnings [None req-f50fcd5b-443a-4364-9872-5d9c84cabc11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:44:53.535221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: warnings.warn(msg) Jun 24 23:44:53.535221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: Jun 24 23:44:53.536424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50fcd5b-443a-4364-9872-5d9c84cabc11 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:53.573186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 750/1497] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:44:53 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:44:53.578289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:53.590402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38963d9a-6345-4cc3-a8a6-7f2c4bccaade admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:53.648869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd873758-e175-4201-8d7e-a2d96399829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:53.649540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd873758-e175-4201-8d7e-a2d96399829c admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.649822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd873758-e175-4201-8d7e-a2d96399829c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.650102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd873758-e175-4201-8d7e-a2d96399829c admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.650788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd873758-e175-4201-8d7e-a2d96399829c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:53.651792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WARNING py.warnings [None req-cd873758-e175-4201-8d7e-a2d96399829c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:44:53.651792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: warnings.warn(msg) Jun 24 23:44:53.651792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: Jun 24 23:44:53.653036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd873758-e175-4201-8d7e-a2d96399829c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:53.675185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 748/1498] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:44:53 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:44:53.680517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fcd5b-443a-4364-9872-5d9c84cabc11 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:53.691574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50fcd5b-443a-4364-9872-5d9c84cabc11 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:53.696077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438129-56ae-4cc2-8560-090433fadc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:53.696761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438129-56ae-4cc2-8560-090433fadc90 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.697046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438129-56ae-4cc2-8560-090433fadc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.697324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e438129-56ae-4cc2-8560-090433fadc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.698044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e438129-56ae-4cc2-8560-090433fadc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:53.699057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WARNING py.warnings [None req-8e438129-56ae-4cc2-8560-090433fadc90 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:44:53.699057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: warnings.warn(msg) Jun 24 23:44:53.699057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: Jun 24 23:44:53.699930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e438129-56ae-4cc2-8560-090433fadc90 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:53.713186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 751/1499] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:44:53.716971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd873758-e175-4201-8d7e-a2d96399829c admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:53.726817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd873758-e175-4201-8d7e-a2d96399829c admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:53.731316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996000e0-42f4-4adf-b44f-5e7b49133f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:53.731963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996000e0-42f4-4adf-b44f-5e7b49133f5e admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.732247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996000e0-42f4-4adf-b44f-5e7b49133f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.732526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996000e0-42f4-4adf-b44f-5e7b49133f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.733278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996000e0-42f4-4adf-b44f-5e7b49133f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:53.734404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WARNING py.warnings [None req-996000e0-42f4-4adf-b44f-5e7b49133f5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:44:53.734404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: warnings.warn(msg) Jun 24 23:44:53.734404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: Jun 24 23:44:53.735273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996000e0-42f4-4adf-b44f-5e7b49133f5e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:53.748335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 749/1500] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:44:53.752748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438129-56ae-4cc2-8560-090433fadc90 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:53.762527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e438129-56ae-4cc2-8560-090433fadc90 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:53.767635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0f2fb-0554-4ab0-8b52-d34d75ac2efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:53.768495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0f2fb-0554-4ab0-8b52-d34d75ac2efa admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.768806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0f2fb-0554-4ab0-8b52-d34d75ac2efa admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.769079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0f2fb-0554-4ab0-8b52-d34d75ac2efa admin admin] PATH_INFO: `/v3/projects/7045cfe1db9e4ad78a8b712f521de14b/users/67ead123b9af45039c33370700f407d6/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.784202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0f2fb-0554-4ab0-8b52-d34d75ac2efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7045cfe1db9e4ad78a8b712f521de14b, user_id=67ead123b9af45039c33370700f407d6, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:53.785277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WARNING py.warnings [None req-e5b0f2fb-0554-4ab0-8b52-d34d75ac2efa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:44:53.785277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: warnings.warn(msg) Jun 24 23:44:53.785277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: Jun 24 23:44:53.787193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0f2fb-0554-4ab0-8b52-d34d75ac2efa admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:53.799982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 752/1501] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:44:53 2022] PUT /identity/v3/projects/7045cfe1db9e4ad78a8b712f521de14b/users/67ead123b9af45039c33370700f407d6/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:44:53.811209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e210d3c-4f65-416c-9bc8-79f4dd7b2f45 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.811498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e210d3c-4f65-416c-9bc8-79f4dd7b2f45 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.812404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e210d3c-4f65-416c-9bc8-79f4dd7b2f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.838341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0e210d3c-4f65-416c-9bc8-79f4dd7b2f45 None None] MFA Rules not processed for user `67ead123b9af45039c33370700f407d6`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:44:53.839541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0e210d3c-4f65-416c-9bc8-79f4dd7b2f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:44:53.908831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 750/1502] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:44:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:44:53.922536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b042b8-67c5-4b13-8ea7-8d96a28c5404 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.922830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b042b8-67c5-4b13-8ea7-8d96a28c5404 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.923111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b042b8-67c5-4b13-8ea7-8d96a28c5404 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.924544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 753/1503] 10.222.0.47 () {54 vars in 963 bytes} [Fri Jun 24 23:44:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:44:53.929280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f0e9c-2b1d-4b6a-9f31-3bd447fe36d7 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:53.929574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f0e9c-2b1d-4b6a-9f31-3bd447fe36d7 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:53.929849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f0e9c-2b1d-4b6a-9f31-3bd447fe36d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:53.947981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-370f0e9c-2b1d-4b6a-9f31-3bd447fe36d7 None None] MFA Rules not processed for user `fb004ddd208849f6a5316c19e9a916aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:44:53.949188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-370f0e9c-2b1d-4b6a-9f31-3bd447fe36d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:44:53.991341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:44:53.999941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 751/1504] 10.222.0.47 () {58 vars in 1060 bytes} [Fri Jun 24 23:44:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:44:54.007627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b042b8-67c5-4b13-8ea7-8d96a28c5404 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:54.018537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b042b8-67c5-4b13-8ea7-8d96a28c5404 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:54.022308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddbb7b-c6a8-4244-9326-abdbefa52f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:54.022939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddbb7b-c6a8-4244-9326-abdbefa52f44 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:54.023236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddbb7b-c6a8-4244-9326-abdbefa52f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:54.023513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddbb7b-c6a8-4244-9326-abdbefa52f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:54.031776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddbb7b-c6a8-4244-9326-abdbefa52f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:54.033253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddbb7b-c6a8-4244-9326-abdbefa52f44 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:54.041866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 754/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:44:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:44:54.084882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a6771-fd77-4ecb-8ce5-76dd3fd31c2a None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:54.085198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a6771-fd77-4ecb-8ce5-76dd3fd31c2a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:54.085486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a6771-fd77-4ecb-8ce5-76dd3fd31c2a None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:54.086773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 752/1506] 10.222.0.47 () {52 vars in 829 bytes} [Fri Jun 24 23:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:44:54.090397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddbb7b-c6a8-4244-9326-abdbefa52f44 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:54.152242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddbb7b-c6a8-4244-9326-abdbefa52f44 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:54.156838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:54.157704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:54.157990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:54.158274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] PATH_INFO: `/v3/users/67ead123b9af45039c33370700f407d6` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:54.159457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ead123b9af45039c33370700f407d6)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:54.160491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WARNING py.warnings [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:44:54.160491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: warnings.warn(msg) Jun 24 23:44:54.160491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: Jun 24 23:44:54.161734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:54.226852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ead123b9af45039c33370700f407d6'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.245341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ead123b9af45039c33370700f407d6'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.246138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ead123b9af45039c33370700f407d6'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.246784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ead123b9af45039c33370700f407d6'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.256296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ead123b9af45039c33370700f407d6'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.264878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:44:54.276339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 755/1507] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:44:54 2022] DELETE /identity/v3/users/67ead123b9af45039c33370700f407d6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:44:54.281439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc416-cf74-43ee-af12-f1f57f3308e0 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:54.281712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc416-cf74-43ee-af12-f1f57f3308e0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:54.281986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc416-cf74-43ee-af12-f1f57f3308e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:54.299046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-78ccc416-cf74-43ee-af12-f1f57f3308e0 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:44:54.300242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-78ccc416-cf74-43ee-af12-f1f57f3308e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:44:54.357151 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 753/1508] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:44:54.366646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:54.368060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:44:54.426999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c80b09-afe6-4691-9e55-3ac0efaf0287 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:54.430630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62c632-027e-4113-931b-0fb974af2f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:54.431288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c632-027e-4113-931b-0fb974af2f9c service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:54.431573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c632-027e-4113-931b-0fb974af2f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:54.431852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62c632-027e-4113-931b-0fb974af2f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:54.441319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62c632-027e-4113-931b-0fb974af2f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:54.442764 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62c632-027e-4113-931b-0fb974af2f9c service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:54.452058 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 756/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:44:54.775929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc416-cf74-43ee-af12-f1f57f3308e0 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:44:54.777481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-78ccc416-cf74-43ee-af12-f1f57f3308e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:44:54.790101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc416-cf74-43ee-af12-f1f57f3308e0 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:44:54.794680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:44:54.795546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:44:54.800915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:44:54.801216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] PATH_INFO: `/v3/projects/7045cfe1db9e4ad78a8b712f521de14b` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:44:54.802479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7045cfe1db9e4ad78a8b712f521de14b)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:44:54.803529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WARNING py.warnings [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:44:54.803529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: warnings.warn(msg) Jun 24 23:44:54.803529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: Jun 24 23:44:54.804780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:44:54.864269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7045cfe1db9e4ad78a8b712f521de14b', 'request_id': 'req-8866f93d-4d86-4aea-b982-80888a319cc3'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.877515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7045cfe1db9e4ad78a8b712f521de14b', 'request_id': 'req-8866f93d-4d86-4aea-b982-80888a319cc3'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.886746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7045cfe1db9e4ad78a8b712f521de14b', 'request_id': 'req-8866f93d-4d86-4aea-b982-80888a319cc3'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.898536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] The token cache is being invalidate because project 7045cfe1db9e4ad78a8b712f521de14b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:44:54.898856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.899620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8866f93d-4d86-4aea-b982-80888a319cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:44:54.901204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 754/1510] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:44:54 2022] DELETE /identity/v3/projects/7045cfe1db9e4ad78a8b712f521de14b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:45:04.447953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=29 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:45:04.895230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:46:30.351408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f26ba-0196-4738-90fe-263f20f0859d None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:30.352002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f26ba-0196-4738-90fe-263f20f0859d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:30.352002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f26ba-0196-4738-90fe-263f20f0859d None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:30.353378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:46:30.358544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5efee-a313-4beb-aee0-bdada1e5fad3 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:30.359030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5efee-a313-4beb-aee0-bdada1e5fad3 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:30.359153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5efee-a313-4beb-aee0-bdada1e5fad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:30.380729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-fab5efee-a313-4beb-aee0-bdada1e5fad3 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:30.381968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-fab5efee-a313-4beb-aee0-bdada1e5fad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:30.446793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:46:30.615549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f26ba-0196-4738-90fe-263f20f0859d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:46:30.617285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d28f26ba-0196-4738-90fe-263f20f0859d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:30.698940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f26ba-0196-4738-90fe-263f20f0859d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:46:30.705985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1612b-bd4d-4b5a-8f35-75b9aac4e889 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:46:30.706638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1612b-bd4d-4b5a-8f35-75b9aac4e889 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:30.706921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1612b-bd4d-4b5a-8f35-75b9aac4e889 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:30.707202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1612b-bd4d-4b5a-8f35-75b9aac4e889 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:30.715782 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1612b-bd4d-4b5a-8f35-75b9aac4e889 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:46:30.718113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1612b-bd4d-4b5a-8f35-75b9aac4e889 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:46:30.729621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 758/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:46:31.585981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49473b-0416-4215-9137-a615cfc1f538 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:31.586516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49473b-0416-4215-9137-a615cfc1f538 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:31.586516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba49473b-0416-4215-9137-a615cfc1f538 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:31.587848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:46:31.593319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcd675-b8b0-41e3-8391-096caf584716 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:31.593752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcd675-b8b0-41e3-8391-096caf584716 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:31.593881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcd675-b8b0-41e3-8391-096caf584716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:31.612548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-affcd675-b8b0-41e3-8391-096caf584716 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:31.613877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-affcd675-b8b0-41e3-8391-096caf584716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:31.621604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:46:31.773140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49473b-0416-4215-9137-a615cfc1f538 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:46:31.788044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba49473b-0416-4215-9137-a615cfc1f538 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:46:31.793456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c08c3-7785-4507-aecd-46966b152e92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:46:31.794099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c08c3-7785-4507-aecd-46966b152e92 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:31.794382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c08c3-7785-4507-aecd-46966b152e92 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:31.795210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c08c3-7785-4507-aecd-46966b152e92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:31.803758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c08c3-7785-4507-aecd-46966b152e92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:46:31.805411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c08c3-7785-4507-aecd-46966b152e92 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:46:31.815438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 757/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:46:32.438652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a24892-cc82-4c5e-a28d-3d4f3f5a0166 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:32.438933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a24892-cc82-4c5e-a28d-3d4f3f5a0166 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:32.439206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a24892-cc82-4c5e-a28d-3d4f3f5a0166 None None] PATH_INFO: `/v3` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:32.440515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 760/1517] 10.222.0.49 () {54 vars in 925 bytes} [Fri Jun 24 23:46:32 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:46:32.470380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c72c40-c499-41cd-88c9-42e1297811ca None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:32.470689 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c72c40-c499-41cd-88c9-42e1297811ca None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:32.470953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c72c40-c499-41cd-88c9-42e1297811ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:32.488529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-06c72c40-c499-41cd-88c9-42e1297811ca None None] MFA Rules not processed for user `a243d444b7ae487cb0331523d4404d19`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:32.489741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-06c72c40-c499-41cd-88c9-42e1297811ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:32.551433 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 758/1518] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Jun 24 23:46:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:46:36.377188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b014f-29aa-4d4e-9204-62d0f91915f4 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:36.377711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b014f-29aa-4d4e-9204-62d0f91915f4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:36.377836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b014f-29aa-4d4e-9204-62d0f91915f4 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:36.379216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 761/1519] 10.222.0.47 () {54 vars in 960 bytes} [Fri Jun 24 23:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:46:36.384658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca91ce-0e41-43f2-9159-1d1274f7be1e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:36.385104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca91ce-0e41-43f2-9159-1d1274f7be1e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:36.385215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca91ce-0e41-43f2-9159-1d1274f7be1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:36.403683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-69ca91ce-0e41-43f2-9159-1d1274f7be1e None None] MFA Rules not processed for user `a243d444b7ae487cb0331523d4404d19`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:36.405044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-69ca91ce-0e41-43f2-9159-1d1274f7be1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:36.413459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 759/1520] 10.222.0.47 () {58 vars in 1057 bytes} [Fri Jun 24 23:46:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:46:36.421335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b014f-29aa-4d4e-9204-62d0f91915f4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:46:36.434633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b014f-29aa-4d4e-9204-62d0f91915f4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:46:36.440465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25190f98-6ea4-42c0-9e94-15727ba6bbe1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:46:36.441127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25190f98-6ea4-42c0-9e94-15727ba6bbe1 service placement] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:36.441406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25190f98-6ea4-42c0-9e94-15727ba6bbe1 service placement] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:36.441675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25190f98-6ea4-42c0-9e94-15727ba6bbe1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:36.449873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25190f98-6ea4-42c0-9e94-15727ba6bbe1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:46:36.452112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25190f98-6ea4-42c0-9e94-15727ba6bbe1 service placement] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:46:36.461955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 762/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:46:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:46:43.060546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc656fb4-1000-4f5d-8fa4-44e5527cb366 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:43.061114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc656fb4-1000-4f5d-8fa4-44e5527cb366 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:43.061114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc656fb4-1000-4f5d-8fa4-44e5527cb366 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:43.062515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:46:43.067311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1ac5b-db2b-4462-9f6a-02f56e1c53a8 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:43.067759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1ac5b-db2b-4462-9f6a-02f56e1c53a8 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:43.067897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1ac5b-db2b-4462-9f6a-02f56e1c53a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:43.086091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b6b1ac5b-db2b-4462-9f6a-02f56e1c53a8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:43.087282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b6b1ac5b-db2b-4462-9f6a-02f56e1c53a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:43.095592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:46:43.250611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc656fb4-1000-4f5d-8fa4-44e5527cb366 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:46:43.265602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc656fb4-1000-4f5d-8fa4-44e5527cb366 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:46:43.271045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce2d43-82f4-4b28-9fe4-bfe8cfd13b73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:46:43.271684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce2d43-82f4-4b28-9fe4-bfe8cfd13b73 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:43.271966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce2d43-82f4-4b28-9fe4-bfe8cfd13b73 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:43.272245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce2d43-82f4-4b28-9fe4-bfe8cfd13b73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:43.280613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce2d43-82f4-4b28-9fe4-bfe8cfd13b73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:46:43.282192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce2d43-82f4-4b28-9fe4-bfe8cfd13b73 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:46:43.291547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 761/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:46:44.138501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5921a5b-0c45-4fbc-aae4-847e63700962 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:44.139146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5921a5b-0c45-4fbc-aae4-847e63700962 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:44.139146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5921a5b-0c45-4fbc-aae4-847e63700962 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:44.140412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 23:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:46:44.145147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48c98-c789-4a73-8d2a-e33349f876d8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:44.145590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48c98-c789-4a73-8d2a-e33349f876d8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:44.145726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48c98-c789-4a73-8d2a-e33349f876d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:44.164798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f6c48c98-c789-4a73-8d2a-e33349f876d8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:44.166004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f6c48c98-c789-4a73-8d2a-e33349f876d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:44.174963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 23:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:46:44.338303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5921a5b-0c45-4fbc-aae4-847e63700962 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:46:44.353687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5921a5b-0c45-4fbc-aae4-847e63700962 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:46:44.359817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476b470-5ad9-4805-bd7b-165404e84e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:46:44.360349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476b470-5ad9-4805-bd7b-165404e84e3a service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:44.360659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476b470-5ad9-4805-bd7b-165404e84e3a service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:44.360940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476b470-5ad9-4805-bd7b-165404e84e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:44.369123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476b470-5ad9-4805-bd7b-165404e84e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:46:44.370679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476b470-5ad9-4805-bd7b-165404e84e3a service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:46:44.379964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 765/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:46:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:46:48.331303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ba380-058b-411f-bd52-e985ee9f6323 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:48.331826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ba380-058b-411f-bd52-e985ee9f6323 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:48.331826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ba380-058b-411f-bd52-e985ee9f6323 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:48.333151 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:46:48.338478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ca62b-d7c7-4c0e-830b-62226410192e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:48.338914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ca62b-d7c7-4c0e-830b-62226410192e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:48.339042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ca62b-d7c7-4c0e-830b-62226410192e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:48.357573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c62ca62b-d7c7-4c0e-830b-62226410192e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:48.358787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c62ca62b-d7c7-4c0e-830b-62226410192e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:48.366857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:46:48.415826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ba380-058b-411f-bd52-e985ee9f6323 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:46:48.417653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ed9ba380-058b-411f-bd52-e985ee9f6323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:48.476592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ba380-058b-411f-bd52-e985ee9f6323 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:46:48.480315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deeb182-c131-4310-b7bf-b8a46fb34f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:46:48.480993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deeb182-c131-4310-b7bf-b8a46fb34f5a service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:48.481295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deeb182-c131-4310-b7bf-b8a46fb34f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:48.482123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deeb182-c131-4310-b7bf-b8a46fb34f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:48.491161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deeb182-c131-4310-b7bf-b8a46fb34f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:46:48.492606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deeb182-c131-4310-b7bf-b8a46fb34f5a service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:46:48.498220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:46:48.503499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 764/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:46:50.552613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c616a-fd16-46d1-ab7c-b9436cec7a05 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:50.553177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c616a-fd16-46d1-ab7c-b9436cec7a05 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:50.553177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c616a-fd16-46d1-ab7c-b9436cec7a05 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:50.554627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:46:50.559845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2124ef3-fbfe-4959-8935-be020b41efeb None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:50.560318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2124ef3-fbfe-4959-8935-be020b41efeb None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:50.560697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2124ef3-fbfe-4959-8935-be020b41efeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:50.579083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f2124ef3-fbfe-4959-8935-be020b41efeb None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:50.580276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f2124ef3-fbfe-4959-8935-be020b41efeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:50.588146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:46:50.623757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c616a-fd16-46d1-ab7c-b9436cec7a05 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:46:50.634465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c616a-fd16-46d1-ab7c-b9436cec7a05 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:46:50.639030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aed2b2-16cd-46ec-b7ca-79d6e3bf2c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:46:50.639660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aed2b2-16cd-46ec-b7ca-79d6e3bf2c80 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:50.639940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aed2b2-16cd-46ec-b7ca-79d6e3bf2c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:50.640214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aed2b2-16cd-46ec-b7ca-79d6e3bf2c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:50.650007 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aed2b2-16cd-46ec-b7ca-79d6e3bf2c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:46:50.651449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aed2b2-16cd-46ec-b7ca-79d6e3bf2c80 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:46:50.666396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 768/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:46:54.161352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc223f-aec8-4655-9c43-83619a7a61cd None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:54.161896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc223f-aec8-4655-9c43-83619a7a61cd None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:54.162004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc223f-aec8-4655-9c43-83619a7a61cd None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:54.163280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:46:54.168445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54208d0-98d1-4dee-9cd2-73b81fb5fbce None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:54.168895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54208d0-98d1-4dee-9cd2-73b81fb5fbce None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:54.169235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54208d0-98d1-4dee-9cd2-73b81fb5fbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:54.187602 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a54208d0-98d1-4dee-9cd2-73b81fb5fbce None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:54.188833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a54208d0-98d1-4dee-9cd2-73b81fb5fbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:54.196545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:46:54.504706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc223f-aec8-4655-9c43-83619a7a61cd None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:46:54.506383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e5dc223f-aec8-4655-9c43-83619a7a61cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:54.564163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc223f-aec8-4655-9c43-83619a7a61cd None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:46:54.567725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae038d7-ac3b-429e-830d-08fab891431b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:46:54.568373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae038d7-ac3b-429e-830d-08fab891431b service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:54.568688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae038d7-ac3b-429e-830d-08fab891431b service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:54.568973 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae038d7-ac3b-429e-830d-08fab891431b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:54.578620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae038d7-ac3b-429e-830d-08fab891431b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:46:54.580041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae038d7-ac3b-429e-830d-08fab891431b service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:46:54.589580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 767/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:46:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:46:56.635854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c6360-4e94-4cc9-b9ab-e72e47717be4 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:56.636420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c6360-4e94-4cc9-b9ab-e72e47717be4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:56.636420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c6360-4e94-4cc9-b9ab-e72e47717be4 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:56.638344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 23:46:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:46:56.643404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5808e-df73-4bb7-bc42-d60ae0aa86a0 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:56.643820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5808e-df73-4bb7-bc42-d60ae0aa86a0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:56.643974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5808e-df73-4bb7-bc42-d60ae0aa86a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:56.663139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-00c5808e-df73-4bb7-bc42-d60ae0aa86a0 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:46:56.664346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-00c5808e-df73-4bb7-bc42-d60ae0aa86a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:56.672177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 23:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:46:56.975450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c6360-4e94-4cc9-b9ab-e72e47717be4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:46:56.977180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-398c6360-4e94-4cc9-b9ab-e72e47717be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:46:56.984571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=65 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:46:56.991351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c6360-4e94-4cc9-b9ab-e72e47717be4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:46:56.994921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79d37-c238-440b-a55e-ef6f3211c7bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:46:56.995553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79d37-c238-440b-a55e-ef6f3211c7bc service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:46:56.995862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79d37-c238-440b-a55e-ef6f3211c7bc service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:46:56.996143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79d37-c238-440b-a55e-ef6f3211c7bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:46:57.005641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b79d37-c238-440b-a55e-ef6f3211c7bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:46:57.007066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b79d37-c238-440b-a55e-ef6f3211c7bc service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:46:57.016978 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 771/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:47:06.657814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=32 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:47:07.012265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:27.048282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e965f30-567e-4438-83cf-1ab4b4299bfc None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.048965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e965f30-567e-4438-83cf-1ab4b4299bfc None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.048965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e965f30-567e-4438-83cf-1ab4b4299bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.060897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900443d-9245-4858-aa66-bf5850da3af2 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.061483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900443d-9245-4858-aa66-bf5850da3af2 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.061797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900443d-9245-4858-aa66-bf5850da3af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.075069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-1e965f30-567e-4438-83cf-1ab4b4299bfc None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.076338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1e965f30-567e-4438-83cf-1ab4b4299bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.087349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1900443d-9245-4858-aa66-bf5850da3af2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.092882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1900443d-9245-4858-aa66-bf5850da3af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.095741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 769/1540] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.104398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17233ff0-18f9-47a9-bece-f6a5765acf50 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.104918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17233ff0-18f9-47a9-bece-f6a5765acf50 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.105370 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17233ff0-18f9-47a9-bece-f6a5765acf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.108416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 772/1541] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:48:27.135198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7be345-cb92-4909-8c39-472785d24d74 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.135766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7be345-cb92-4909-8c39-472785d24d74 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.136217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7be345-cb92-4909-8c39-472785d24d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.139187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-17233ff0-18f9-47a9-bece-f6a5765acf50 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.140358 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-17233ff0-18f9-47a9-bece-f6a5765acf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.164607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 770/1542] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.166243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17233ff0-18f9-47a9-bece-f6a5765acf50 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.171494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-db7be345-cb92-4909-8c39-472785d24d74 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.172891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-db7be345-cb92-4909-8c39-472785d24d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.183005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17233ff0-18f9-47a9-bece-f6a5765acf50 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.186972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 773/1543] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.194480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7be345-cb92-4909-8c39-472785d24d74 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.195499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c8ee7-a40f-44de-bdc4-56e1969c26c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.196196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c8ee7-a40f-44de-bdc4-56e1969c26c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.196477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c8ee7-a40f-44de-bdc4-56e1969c26c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.196784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c8ee7-a40f-44de-bdc4-56e1969c26c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.197543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c8ee7-a40f-44de-bdc4-56e1969c26c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:27.198468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c8ee7-a40f-44de-bdc4-56e1969c26c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.200883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c8ee7-a40f-44de-bdc4-56e1969c26c9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.206195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7be345-cb92-4909-8c39-472785d24d74 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.218071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.218680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.218966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.219243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.220000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:27.220312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.221421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WARNING py.warnings [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:48:27.221421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: warnings.warn(msg) Jun 24 23:48:27.221421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: Jun 24 23:48:27.222345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.229403 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 771/1544] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:48:27.232254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c04a0d-16c1-47fa-842b-1c60fdd383fe None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.232781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c04a0d-16c1-47fa-842b-1c60fdd383fe None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.233422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c04a0d-16c1-47fa-842b-1c60fdd383fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.249214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 774/1545] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:48:27.266064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.271581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-30c04a0d-16c1-47fa-842b-1c60fdd383fe None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.277231 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-30c04a0d-16c1-47fa-842b-1c60fdd383fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.282866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e29443-9de1-48ae-af14-b0445a6e25e0 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.298913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 772/1546] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.300215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c04a0d-16c1-47fa-842b-1c60fdd383fe None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.300785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a958ca-f54a-40ad-8e5f-1e7f2c3269af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.304610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a958ca-f54a-40ad-8e5f-1e7f2c3269af admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.304610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a958ca-f54a-40ad-8e5f-1e7f2c3269af admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.304610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a958ca-f54a-40ad-8e5f-1e7f2c3269af admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.304610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a958ca-f54a-40ad-8e5f-1e7f2c3269af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.304962 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a958ca-f54a-40ad-8e5f-1e7f2c3269af admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.327829 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c04a0d-16c1-47fa-842b-1c60fdd383fe None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.345212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8d32d-e124-4802-89ad-e18a0e248804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.346145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8d32d-e124-4802-89ad-e18a0e248804 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.346451 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8d32d-e124-4802-89ad-e18a0e248804 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.346723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8d32d-e124-4802-89ad-e18a0e248804 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.347627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8d32d-e124-4802-89ad-e18a0e248804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.348865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WARNING py.warnings [None req-fbd8d32d-e124-4802-89ad-e18a0e248804 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:48:27.348865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: warnings.warn(msg) Jun 24 23:48:27.348865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: Jun 24 23:48:27.350298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8d32d-e124-4802-89ad-e18a0e248804 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.374420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 775/1547] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:27.385430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134cc63-35c5-49d8-8c29-e47b1e902487 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.386351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134cc63-35c5-49d8-8c29-e47b1e902487 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.386627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134cc63-35c5-49d8-8c29-e47b1e902487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.415453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6134cc63-35c5-49d8-8c29-e47b1e902487 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.416780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6134cc63-35c5-49d8-8c29-e47b1e902487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.439348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 773/1548] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:27.440532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b00367-6ecd-4f32-992c-1731ff740755 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.440860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b00367-6ecd-4f32-992c-1731ff740755 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.441143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b00367-6ecd-4f32-992c-1731ff740755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.468909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-03b00367-6ecd-4f32-992c-1731ff740755 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.470709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-03b00367-6ecd-4f32-992c-1731ff740755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.489466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 776/1549] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.490300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134cc63-35c5-49d8-8c29-e47b1e902487 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.500266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134cc63-35c5-49d8-8c29-e47b1e902487 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.543788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 774/1550] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.545181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1cc27-30ee-493b-ad58-535fd332ae0e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.545475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1cc27-30ee-493b-ad58-535fd332ae0e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.545762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1cc27-30ee-493b-ad58-535fd332ae0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.560561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba2f86-1737-4279-81fc-029e01dcc136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.561267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba2f86-1737-4279-81fc-029e01dcc136 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.561560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba2f86-1737-4279-81fc-029e01dcc136 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.561841 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba2f86-1737-4279-81fc-029e01dcc136 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.562543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba2f86-1737-4279-81fc-029e01dcc136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.563579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WARNING py.warnings [None req-ceba2f86-1737-4279-81fc-029e01dcc136 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:48:27.563579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: warnings.warn(msg) Jun 24 23:48:27.563579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: Jun 24 23:48:27.563955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d2f1cc27-30ee-493b-ad58-535fd332ae0e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.564910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba2f86-1737-4279-81fc-029e01dcc136 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.565199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d2f1cc27-30ee-493b-ad58-535fd332ae0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.573280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 775/1551] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.574042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1cc27-30ee-493b-ad58-535fd332ae0e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.584752 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1cc27-30ee-493b-ad58-535fd332ae0e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.588992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 777/1552] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:27.588992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9eef5-f86a-44e5-b41e-2b9c0ba7e30a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.588992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9eef5-f86a-44e5-b41e-2b9c0ba7e30a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.588992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9eef5-f86a-44e5-b41e-2b9c0ba7e30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.589844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fdd20-7a2d-4454-ae8f-dc8abb9c1a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.590509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdd20-7a2d-4454-ae8f-dc8abb9c1a2f admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.591287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdd20-7a2d-4454-ae8f-dc8abb9c1a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.591563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdd20-7a2d-4454-ae8f-dc8abb9c1a2f admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.592313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fdd20-7a2d-4454-ae8f-dc8abb9c1a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.594314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fdd20-7a2d-4454-ae8f-dc8abb9c1a2f admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.607821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f6b9eef5-f86a-44e5-b41e-2b9c0ba7e30a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.608803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f6b9eef5-f86a-44e5-b41e-2b9c0ba7e30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.619220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 776/1553] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/users => generated 282 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:27.619220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a49078-0322-4f61-9822-46cf15cb9c34 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.619220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a49078-0322-4f61-9822-46cf15cb9c34 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.619220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a49078-0322-4f61-9822-46cf15cb9c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.621785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 778/1554] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.622638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9eef5-f86a-44e5-b41e-2b9c0ba7e30a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.632379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9eef5-f86a-44e5-b41e-2b9c0ba7e30a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.636915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58068e11-72d1-4b24-801b-adbcf987dc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.637615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58068e11-72d1-4b24-801b-adbcf987dc22 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.637932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58068e11-72d1-4b24-801b-adbcf987dc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.638199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58068e11-72d1-4b24-801b-adbcf987dc22 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.638936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58068e11-72d1-4b24-801b-adbcf987dc22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:27.639250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58068e11-72d1-4b24-801b-adbcf987dc22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.640880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58068e11-72d1-4b24-801b-adbcf987dc22 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.642446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a6a49078-0322-4f61-9822-46cf15cb9c34 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.643650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a6a49078-0322-4f61-9822-46cf15cb9c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.652287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 777/1555] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.660346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 779/1556] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:27.660523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405688ff-7831-4c2c-9cfb-4ee9a244a9be None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.660523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405688ff-7831-4c2c-9cfb-4ee9a244a9be None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.660523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405688ff-7831-4c2c-9cfb-4ee9a244a9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.660906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009de92-a579-453d-80b1-16076c58786a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.660906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009de92-a579-453d-80b1-16076c58786a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.660906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009de92-a579-453d-80b1-16076c58786a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.677446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-405688ff-7831-4c2c-9cfb-4ee9a244a9be None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.678015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-e009de92-a579-453d-80b1-16076c58786a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.678686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-405688ff-7831-4c2c-9cfb-4ee9a244a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.679207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e009de92-a579-453d-80b1-16076c58786a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.687837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 778/1557] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.688308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 780/1558] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.689180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009de92-a579-453d-80b1-16076c58786a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.693874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc2d1-4653-4a32-af88-394631f0993a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.694080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc2d1-4653-4a32-af88-394631f0993a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.694237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc2d1-4653-4a32-af88-394631f0993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.701001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009de92-a579-453d-80b1-16076c58786a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.706592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6068-ed5a-46cb-90a3-d4c2892d2740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.707428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6068-ed5a-46cb-90a3-d4c2892d2740 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.707741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6068-ed5a-46cb-90a3-d4c2892d2740 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.708066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6068-ed5a-46cb-90a3-d4c2892d2740 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.709030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae6068-ed5a-46cb-90a3-d4c2892d2740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.710964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae6068-ed5a-46cb-90a3-d4c2892d2740 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.715037 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-276bc2d1-4653-4a32-af88-394631f0993a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.716887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-276bc2d1-4653-4a32-af88-394631f0993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.726310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 781/1559] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:27.726310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6068-ed5a-46cb-90a3-d4c2892d2740 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.727990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 779/1560] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.729333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e890e6-1f89-46b4-b153-d5ffaf4b0972 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.729626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e890e6-1f89-46b4-b153-d5ffaf4b0972 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.729908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e890e6-1f89-46b4-b153-d5ffaf4b0972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.733982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:27.738134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6068-ed5a-46cb-90a3-d4c2892d2740 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.743887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265b8f0-c542-448a-a6ca-f5bfc37424f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.744574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265b8f0-c542-448a-a6ca-f5bfc37424f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.744888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265b8f0-c542-448a-a6ca-f5bfc37424f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.745168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265b8f0-c542-448a-a6ca-f5bfc37424f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.745899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265b8f0-c542-448a-a6ca-f5bfc37424f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.748052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265b8f0-c542-448a-a6ca-f5bfc37424f0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.749210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-59e890e6-1f89-46b4-b153-d5ffaf4b0972 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.750408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-59e890e6-1f89-46b4-b153-d5ffaf4b0972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.758443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 782/1561] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:27.759492 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3c193-add7-4deb-bc53-4112cefd97b2 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.759787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3c193-add7-4deb-bc53-4112cefd97b2 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.760072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3c193-add7-4deb-bc53-4112cefd97b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.760655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 780/1562] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.762145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e890e6-1f89-46b4-b153-d5ffaf4b0972 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.772472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e890e6-1f89-46b4-b153-d5ffaf4b0972 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.777563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be9dd6-7d8f-4cd7-83d7-334bf60904cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.778285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9dd6-7d8f-4cd7-83d7-334bf60904cb admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.778574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9dd6-7d8f-4cd7-83d7-334bf60904cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.778856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be9dd6-7d8f-4cd7-83d7-334bf60904cb admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.779553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be9dd6-7d8f-4cd7-83d7-334bf60904cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.780083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1fb3c193-add7-4deb-bc53-4112cefd97b2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.781304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1fb3c193-add7-4deb-bc53-4112cefd97b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.781539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be9dd6-7d8f-4cd7-83d7-334bf60904cb admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.789679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 783/1563] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.790878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3c193-add7-4deb-bc53-4112cefd97b2 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.801455 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb3c193-add7-4deb-bc53-4112cefd97b2 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.806035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a15cfb-33ff-4668-82ff-cd688fdd2bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.806682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a15cfb-33ff-4668-82ff-cd688fdd2bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.806965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a15cfb-33ff-4668-82ff-cd688fdd2bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.807242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a15cfb-33ff-4668-82ff-cd688fdd2bf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.811382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 781/1564] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:27.811533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a15cfb-33ff-4668-82ff-cd688fdd2bf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:27.811533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a15cfb-33ff-4668-82ff-cd688fdd2bf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.811692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3abd5-1ce8-4992-9f02-fa1b11f99dff None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.811778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a15cfb-33ff-4668-82ff-cd688fdd2bf5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.811854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3abd5-1ce8-4992-9f02-fa1b11f99dff None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.811854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3abd5-1ce8-4992-9f02-fa1b11f99dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.822765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 784/1565] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:27.823470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a15cfb-33ff-4668-82ff-cd688fdd2bf5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.834261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a15cfb-33ff-4668-82ff-cd688fdd2bf5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.836113 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d7e3abd5-1ce8-4992-9f02-fa1b11f99dff None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:27.837459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d7e3abd5-1ce8-4992-9f02-fa1b11f99dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:27.866611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:27.892647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a023e-50d5-4267-98f1-c29d1fe57b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.893349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a023e-50d5-4267-98f1-c29d1fe57b21 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.893635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a023e-50d5-4267-98f1-c29d1fe57b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.893914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a023e-50d5-4267-98f1-c29d1fe57b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.894643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a023e-50d5-4267-98f1-c29d1fe57b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.896249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a023e-50d5-4267-98f1-c29d1fe57b21 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.898097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 782/1566] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:27.898926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3abd5-1ce8-4992-9f02-fa1b11f99dff None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.908559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3abd5-1ce8-4992-9f02-fa1b11f99dff None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.910867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 785/1567] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:27.911553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a023e-50d5-4267-98f1-c29d1fe57b21 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.915879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e1d14-fa7e-4430-b5c9-6ca839076c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.916540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e1d14-fa7e-4430-b5c9-6ca839076c88 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.916875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e1d14-fa7e-4430-b5c9-6ca839076c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.917156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e1d14-fa7e-4430-b5c9-6ca839076c88 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.917892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e1d14-fa7e-4430-b5c9-6ca839076c88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:27.918388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e1d14-fa7e-4430-b5c9-6ca839076c88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.921124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e1d14-fa7e-4430-b5c9-6ca839076c88 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.923435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a023e-50d5-4267-98f1-c29d1fe57b21 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.928013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859eb38-2dd2-40c4-9607-1e6d47b25689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.928691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859eb38-2dd2-40c4-9607-1e6d47b25689 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.928983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859eb38-2dd2-40c4-9607-1e6d47b25689 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.929261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859eb38-2dd2-40c4-9607-1e6d47b25689 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.929974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859eb38-2dd2-40c4-9607-1e6d47b25689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.942813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859eb38-2dd2-40c4-9607-1e6d47b25689 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.943022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 783/1568] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:27.943022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e1d14-fa7e-4430-b5c9-6ca839076c88 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.943181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 786/1569] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:27.944422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859eb38-2dd2-40c4-9607-1e6d47b25689 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.952659 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e1d14-fa7e-4430-b5c9-6ca839076c88 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.954283 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859eb38-2dd2-40c4-9607-1e6d47b25689 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.957541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80a54c-390d-4f2e-a6e7-9da7ec259295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.958198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80a54c-390d-4f2e-a6e7-9da7ec259295 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.958583 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80a54c-390d-4f2e-a6e7-9da7ec259295 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.958862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80a54c-390d-4f2e-a6e7-9da7ec259295 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.959005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32f2b2-cc22-4155-a473-ab1329b13155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.959587 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80a54c-390d-4f2e-a6e7-9da7ec259295 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:27.959710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32f2b2-cc22-4155-a473-ab1329b13155 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.959892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80a54c-390d-4f2e-a6e7-9da7ec259295 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.959990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32f2b2-cc22-4155-a473-ab1329b13155 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.960167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32f2b2-cc22-4155-a473-ab1329b13155 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.960907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32f2b2-cc22-4155-a473-ab1329b13155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:27.961279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32f2b2-cc22-4155-a473-ab1329b13155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.961508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80a54c-390d-4f2e-a6e7-9da7ec259295 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.962853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32f2b2-cc22-4155-a473-ab1329b13155 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.974238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 784/1570] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:27.975303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80a54c-390d-4f2e-a6e7-9da7ec259295 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.975445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 787/1571] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:27.975933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32f2b2-cc22-4155-a473-ab1329b13155 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:27.987084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32f2b2-cc22-4155-a473-ab1329b13155 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.991644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43238f-16cc-49c5-8825-8d8482be7ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.992286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43238f-16cc-49c5-8825-8d8482be7ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.992567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43238f-16cc-49c5-8825-8d8482be7ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.992874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43238f-16cc-49c5-8825-8d8482be7ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.993092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80a54c-390d-4f2e-a6e7-9da7ec259295 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:27.993555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43238f-16cc-49c5-8825-8d8482be7ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:27.995439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43238f-16cc-49c5-8825-8d8482be7ac4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:27.997901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e34c8-3352-43ab-8699-cac2380c7bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:27.998555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e34c8-3352-43ab-8699-cac2380c7bec admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:27.998837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e34c8-3352-43ab-8699-cac2380c7bec admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:27.999108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e34c8-3352-43ab-8699-cac2380c7bec admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:27.999812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e34c8-3352-43ab-8699-cac2380c7bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.001740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e34c8-3352-43ab-8699-cac2380c7bec admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.025356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 788/1572] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.025356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43238f-16cc-49c5-8825-8d8482be7ac4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.025664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 785/1573] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:27 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.025664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e34c8-3352-43ab-8699-cac2380c7bec admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.034634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e34c8-3352-43ab-8699-cac2380c7bec admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.088378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43238f-16cc-49c5-8825-8d8482be7ac4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.090735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0c036-b21a-45e7-80cf-cbba9fa8d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.092903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0c036-b21a-45e7-80cf-cbba9fa8d87a admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.093191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0c036-b21a-45e7-80cf-cbba9fa8d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.093472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0c036-b21a-45e7-80cf-cbba9fa8d87a admin admin] PATH_INFO: `/v3/projects/5ccf88e235fa45d1b92c7c5612bfe140/users/2e78cf97bac4484f93c7359b86a26628/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.094220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc278d1-6978-4f11-a18f-ffd30d9d07d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.094882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc278d1-6978-4f11-a18f-ffd30d9d07d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.095163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc278d1-6978-4f11-a18f-ffd30d9d07d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.095435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc278d1-6978-4f11-a18f-ffd30d9d07d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.096159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc278d1-6978-4f11-a18f-ffd30d9d07d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:28.096467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc278d1-6978-4f11-a18f-ffd30d9d07d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.098106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc278d1-6978-4f11-a18f-ffd30d9d07d2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.108129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0c036-b21a-45e7-80cf-cbba9fa8d87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccf88e235fa45d1b92c7c5612bfe140, user_id=2e78cf97bac4484f93c7359b86a26628, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.109230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WARNING py.warnings [None req-e9a0c036-b21a-45e7-80cf-cbba9fa8d87a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:48:28.109230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: warnings.warn(msg) Jun 24 23:48:28.109230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: Jun 24 23:48:28.109565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 789/1574] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:28.110345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc278d1-6978-4f11-a18f-ffd30d9d07d2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.111433 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0c036-b21a-45e7-80cf-cbba9fa8d87a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.123457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc278d1-6978-4f11-a18f-ffd30d9d07d2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.130356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 786/1575] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:28 2022] PUT /identity/v3/projects/5ccf88e235fa45d1b92c7c5612bfe140/users/2e78cf97bac4484f93c7359b86a26628/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:28.130942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0c036-b21a-45e7-80cf-cbba9fa8d87a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.139162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:28.185105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a92fbb-d8f4-41fb-8e40-7486b186eed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.185771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92fbb-d8f4-41fb-8e40-7486b186eed4 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.186061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92fbb-d8f4-41fb-8e40-7486b186eed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.186343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a92fbb-d8f4-41fb-8e40-7486b186eed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.187037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a92fbb-d8f4-41fb-8e40-7486b186eed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.188998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a92fbb-d8f4-41fb-8e40-7486b186eed4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.194016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0c036-b21a-45e7-80cf-cbba9fa8d87a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.198602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742cdd5-bab2-47a0-9c1c-1a0e720fdeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.199486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742cdd5-bab2-47a0-9c1c-1a0e720fdeee admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.199764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742cdd5-bab2-47a0-9c1c-1a0e720fdeee admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.200038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742cdd5-bab2-47a0-9c1c-1a0e720fdeee admin admin] PATH_INFO: `/v3/projects/55c71ddb236848a387dd9949b44d4bd2/users/25ed640040f94fc085bc565e3519f231/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.215102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742cdd5-bab2-47a0-9c1c-1a0e720fdeee admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c71ddb236848a387dd9949b44d4bd2, user_id=25ed640040f94fc085bc565e3519f231, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.215880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 790/1576] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.216724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a92fbb-d8f4-41fb-8e40-7486b186eed4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.217982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742cdd5-bab2-47a0-9c1c-1a0e720fdeee admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.221119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:28.232955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 787/1577] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:28 2022] PUT /identity/v3/projects/55c71ddb236848a387dd9949b44d4bd2/users/25ed640040f94fc085bc565e3519f231/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:28.234035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742cdd5-bab2-47a0-9c1c-1a0e720fdeee admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.283760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a92fbb-d8f4-41fb-8e40-7486b186eed4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.296181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742cdd5-bab2-47a0-9c1c-1a0e720fdeee admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.300963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f88d65-51bd-45d8-ab6f-a4a71767135c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.301633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f88d65-51bd-45d8-ab6f-a4a71767135c admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.301916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f88d65-51bd-45d8-ab6f-a4a71767135c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.302193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f88d65-51bd-45d8-ab6f-a4a71767135c admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.302886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f88d65-51bd-45d8-ab6f-a4a71767135c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.304806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f88d65-51bd-45d8-ab6f-a4a71767135c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.331226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 788/1578] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.332269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f88d65-51bd-45d8-ab6f-a4a71767135c admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.340124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850fc7b-da6b-4ef5-a421-d18504f26a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.340841 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850fc7b-da6b-4ef5-a421-d18504f26a0c admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.341127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850fc7b-da6b-4ef5-a421-d18504f26a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.341420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850fc7b-da6b-4ef5-a421-d18504f26a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.342174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850fc7b-da6b-4ef5-a421-d18504f26a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.344115 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850fc7b-da6b-4ef5-a421-d18504f26a0c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.369505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 791/1579] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/projects => generated 393 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.370337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850fc7b-da6b-4ef5-a421-d18504f26a0c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.396304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f88d65-51bd-45d8-ab6f-a4a71767135c admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.417451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:28.443484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850fc7b-da6b-4ef5-a421-d18504f26a0c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.448490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f046-94af-4a73-901c-948b3c9d4e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.449273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f046-94af-4a73-901c-948b3c9d4e0c admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.449462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f046-94af-4a73-901c-948b3c9d4e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.449734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f046-94af-4a73-901c-948b3c9d4e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.451035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475f046-94af-4a73-901c-948b3c9d4e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.452668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475f046-94af-4a73-901c-948b3c9d4e0c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.455972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6e6c9-2dfe-4551-a8b1-8d5fb2ac4e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.456676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6e6c9-2dfe-4551-a8b1-8d5fb2ac4e99 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.456966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6e6c9-2dfe-4551-a8b1-8d5fb2ac4e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.457247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6e6c9-2dfe-4551-a8b1-8d5fb2ac4e99 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.458255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6e6c9-2dfe-4551-a8b1-8d5fb2ac4e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.460105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6e6c9-2dfe-4551-a8b1-8d5fb2ac4e99 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.463006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 792/1580] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:28.463883 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f046-94af-4a73-901c-948b3c9d4e0c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.474859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f046-94af-4a73-901c-948b3c9d4e0c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.479435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca359931-96cb-48e3-925c-cdfe5621af64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.480078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca359931-96cb-48e3-925c-cdfe5621af64 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.480362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca359931-96cb-48e3-925c-cdfe5621af64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.480503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 789/1581] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.480728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca359931-96cb-48e3-925c-cdfe5621af64 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.481506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca359931-96cb-48e3-925c-cdfe5621af64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.482500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2adcfc-2981-47d3-997c-30b6f515d514 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.482796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2adcfc-2981-47d3-997c-30b6f515d514 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.483080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2adcfc-2981-47d3-997c-30b6f515d514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.483260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca359931-96cb-48e3-925c-cdfe5621af64 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.500959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-bf2adcfc-2981-47d3-997c-30b6f515d514 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:28.502120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-bf2adcfc-2981-47d3-997c-30b6f515d514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:28.509759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 790/1582] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:28.513796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 793/1583] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.513796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ae583-9f2f-4283-93ce-ff9ea5da4293 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.513796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ae583-9f2f-4283-93ce-ff9ea5da4293 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.513796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ae583-9f2f-4283-93ce-ff9ea5da4293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.514223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2adcfc-2981-47d3-997c-30b6f515d514 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.523879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2adcfc-2981-47d3-997c-30b6f515d514 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.532505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:28.539495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d75ae583-9f2f-4283-93ce-ff9ea5da4293 None None] MFA Rules not processed for user `25ed640040f94fc085bc565e3519f231`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:28.540677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d75ae583-9f2f-4283-93ce-ff9ea5da4293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:28.583081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97364eb1-8e39-4381-8485-e275d9bb4909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.583759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97364eb1-8e39-4381-8485-e275d9bb4909 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.584046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97364eb1-8e39-4381-8485-e275d9bb4909 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.584333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97364eb1-8e39-4381-8485-e275d9bb4909 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.585059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97364eb1-8e39-4381-8485-e275d9bb4909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.587021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97364eb1-8e39-4381-8485-e275d9bb4909 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.608258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 791/1584] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.609361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97364eb1-8e39-4381-8485-e275d9bb4909 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.612726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 794/1585] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:28.612726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ae583-9f2f-4283-93ce-ff9ea5da4293 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.618839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97364eb1-8e39-4381-8485-e275d9bb4909 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.624160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97120a-de58-475a-8fb0-1b12912add68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.624857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97120a-de58-475a-8fb0-1b12912add68 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.625143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97120a-de58-475a-8fb0-1b12912add68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.625422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97120a-de58-475a-8fb0-1b12912add68 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.625754 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ae583-9f2f-4283-93ce-ff9ea5da4293 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.626105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97120a-de58-475a-8fb0-1b12912add68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.628016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97120a-de58-475a-8fb0-1b12912add68 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.630286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958602af-9a21-4944-bdcf-cdcf61ce249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.630933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958602af-9a21-4944-bdcf-cdcf61ce249f admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.631220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958602af-9a21-4944-bdcf-cdcf61ce249f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.631496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958602af-9a21-4944-bdcf-cdcf61ce249f admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.632176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958602af-9a21-4944-bdcf-cdcf61ce249f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.634143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958602af-9a21-4944-bdcf-cdcf61ce249f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.649040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 792/1586] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/users => generated 286 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.649768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97120a-de58-475a-8fb0-1b12912add68 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.655581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 795/1587] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.656454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958602af-9a21-4944-bdcf-cdcf61ce249f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.661705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97120a-de58-475a-8fb0-1b12912add68 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.665917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958602af-9a21-4944-bdcf-cdcf61ce249f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.666290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ca755-cb8a-4cd4-9351-1151d8c25d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.667070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca755-cb8a-4cd4-9351-1151d8c25d4c admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.667357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca755-cb8a-4cd4-9351-1151d8c25d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.667663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca755-cb8a-4cd4-9351-1151d8c25d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.668423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ca755-cb8a-4cd4-9351-1151d8c25d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.670848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ca755-cb8a-4cd4-9351-1151d8c25d4c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.671672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6ed4f-dc93-4d1c-89de-21081611a88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.672319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6ed4f-dc93-4d1c-89de-21081611a88b admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.672621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6ed4f-dc93-4d1c-89de-21081611a88b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.672909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6ed4f-dc93-4d1c-89de-21081611a88b admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.673626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6ed4f-dc93-4d1c-89de-21081611a88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.675268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6ed4f-dc93-4d1c-89de-21081611a88b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.681916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 793/1588] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:28.682856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ca755-cb8a-4cd4-9351-1151d8c25d4c admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.692545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 796/1589] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:28.692545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6ed4f-dc93-4d1c-89de-21081611a88b admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.697635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ca755-cb8a-4cd4-9351-1151d8c25d4c admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.703770 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e212cb3-72b7-4b7f-a469-9dde30222222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.704449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e212cb3-72b7-4b7f-a469-9dde30222222 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.704774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e212cb3-72b7-4b7f-a469-9dde30222222 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.705058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e212cb3-72b7-4b7f-a469-9dde30222222 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.705763 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e212cb3-72b7-4b7f-a469-9dde30222222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.707883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e212cb3-72b7-4b7f-a469-9dde30222222 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.732243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 794/1590] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.732243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e212cb3-72b7-4b7f-a469-9dde30222222 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.739946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e212cb3-72b7-4b7f-a469-9dde30222222 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.745985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cae95-ed0c-42da-ad6f-6abcaaea48a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.746635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cae95-ed0c-42da-ad6f-6abcaaea48a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.746962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cae95-ed0c-42da-ad6f-6abcaaea48a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.747315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cae95-ed0c-42da-ad6f-6abcaaea48a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.748148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cae95-ed0c-42da-ad6f-6abcaaea48a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.749886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cae95-ed0c-42da-ad6f-6abcaaea48a8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.754033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6ed4f-dc93-4d1c-89de-21081611a88b admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.757685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cd821-3443-4ec2-81d8-415929f06930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.758340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cd821-3443-4ec2-81d8-415929f06930 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.758625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cd821-3443-4ec2-81d8-415929f06930 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.758903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cd821-3443-4ec2-81d8-415929f06930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.764711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 795/1591] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:28.764711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cae95-ed0c-42da-ad6f-6abcaaea48a8 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.771994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cd821-3443-4ec2-81d8-415929f06930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.773719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cd821-3443-4ec2-81d8-415929f06930 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.773952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cae95-ed0c-42da-ad6f-6abcaaea48a8 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.779623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb76cd5-8bae-4ec0-9291-89acbd51c270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.780314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb76cd5-8bae-4ec0-9291-89acbd51c270 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.780627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb76cd5-8bae-4ec0-9291-89acbd51c270 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.780910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb76cd5-8bae-4ec0-9291-89acbd51c270 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.781624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb76cd5-8bae-4ec0-9291-89acbd51c270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.783516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb76cd5-8bae-4ec0-9291-89acbd51c270 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.785086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 797/1592] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:28.785820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cd821-3443-4ec2-81d8-415929f06930 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.795553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cd821-3443-4ec2-81d8-415929f06930 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.800898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac914778-8b46-4c1a-9558-8e55d245192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.801594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac914778-8b46-4c1a-9558-8e55d245192d admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.801931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac914778-8b46-4c1a-9558-8e55d245192d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.802211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac914778-8b46-4c1a-9558-8e55d245192d admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.803024 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac914778-8b46-4c1a-9558-8e55d245192d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.804684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac914778-8b46-4c1a-9558-8e55d245192d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.813376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 796/1593] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:28 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:28.818776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 798/1594] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:28.818776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac914778-8b46-4c1a-9558-8e55d245192d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.819128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb76cd5-8bae-4ec0-9291-89acbd51c270 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.824340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:28.830890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac914778-8b46-4c1a-9558-8e55d245192d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.897019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb76cd5-8bae-4ec0-9291-89acbd51c270 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.902099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd21642-ebb3-4309-9c89-98c82176fa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.902688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f81767-b85d-4f60-822f-45849fcbd7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.903027 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd21642-ebb3-4309-9c89-98c82176fa8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.903576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f81767-b85d-4f60-822f-45849fcbd7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.903935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f81767-b85d-4f60-822f-45849fcbd7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.904232 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd21642-ebb3-4309-9c89-98c82176fa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.904490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f81767-b85d-4f60-822f-45849fcbd7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.904642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd21642-ebb3-4309-9c89-98c82176fa8b admin admin] PATH_INFO: `/v3/projects/4921002ea57c4f54a7a3b69d802a6430/users/151db87b1e7248d882108519fab1d938/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.905398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f81767-b85d-4f60-822f-45849fcbd7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.907078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f81767-b85d-4f60-822f-45849fcbd7a9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.922353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 797/1595] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:28.922353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f81767-b85d-4f60-822f-45849fcbd7a9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.930386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd21642-ebb3-4309-9c89-98c82176fa8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4921002ea57c4f54a7a3b69d802a6430, user_id=151db87b1e7248d882108519fab1d938, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.933483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd21642-ebb3-4309-9c89-98c82176fa8b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.935634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f81767-b85d-4f60-822f-45849fcbd7a9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.941404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba583704-aea1-491c-a170-d68dcb333ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:28.942224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba583704-aea1-491c-a170-d68dcb333ced admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:28.942635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba583704-aea1-491c-a170-d68dcb333ced admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:28.942966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba583704-aea1-491c-a170-d68dcb333ced admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:28.943647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba583704-aea1-491c-a170-d68dcb333ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:28.945652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba583704-aea1-491c-a170-d68dcb333ced admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:28.951573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 799/1596] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:28 2022] PUT /identity/v3/projects/4921002ea57c4f54a7a3b69d802a6430/users/151db87b1e7248d882108519fab1d938/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:28.952384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd21642-ebb3-4309-9c89-98c82176fa8b admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.967068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 798/1597] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:28.969711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba583704-aea1-491c-a170-d68dcb333ced admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:28.982311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba583704-aea1-491c-a170-d68dcb333ced admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:28.988837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:29.030989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd21642-ebb3-4309-9c89-98c82176fa8b admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.035878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec5518-8d1b-402c-a103-6c4c45351349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.036557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec5518-8d1b-402c-a103-6c4c45351349 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.036877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec5518-8d1b-402c-a103-6c4c45351349 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.037160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec5518-8d1b-402c-a103-6c4c45351349 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.037957 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec5518-8d1b-402c-a103-6c4c45351349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.039614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec5518-8d1b-402c-a103-6c4c45351349 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.052543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 800/1598] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:29.053401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec5518-8d1b-402c-a103-6c4c45351349 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.067890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec5518-8d1b-402c-a103-6c4c45351349 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.068083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c41e97-1fce-4ed6-b02d-f7284be37b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.068775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c41e97-1fce-4ed6-b02d-f7284be37b5b admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.069073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c41e97-1fce-4ed6-b02d-f7284be37b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.069680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c41e97-1fce-4ed6-b02d-f7284be37b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.070473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c41e97-1fce-4ed6-b02d-f7284be37b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.072368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c41e97-1fce-4ed6-b02d-f7284be37b5b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.073128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17631b-3476-4a92-9adc-f14884c3d378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.073801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17631b-3476-4a92-9adc-f14884c3d378 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.074092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17631b-3476-4a92-9adc-f14884c3d378 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.074468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17631b-3476-4a92-9adc-f14884c3d378 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.075610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17631b-3476-4a92-9adc-f14884c3d378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.078750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17631b-3476-4a92-9adc-f14884c3d378 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.084515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 799/1599] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:29.086452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c41e97-1fce-4ed6-b02d-f7284be37b5b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.107090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c41e97-1fce-4ed6-b02d-f7284be37b5b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.110109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 801/1600] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:29 2022] POST /identity/v3/users => generated 289 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:29.111550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17631b-3476-4a92-9adc-f14884c3d378 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.118841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d041f2-5af6-4425-8df2-8a942edce50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.119656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d041f2-5af6-4425-8df2-8a942edce50a admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.120020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d041f2-5af6-4425-8df2-8a942edce50a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.120514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d041f2-5af6-4425-8df2-8a942edce50a admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.122935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d041f2-5af6-4425-8df2-8a942edce50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.125439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d041f2-5af6-4425-8df2-8a942edce50a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.126650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17631b-3476-4a92-9adc-f14884c3d378 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.133863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2068a8b-20f3-4786-9962-076c5327a598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.134717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2068a8b-20f3-4786-9962-076c5327a598 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.135156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2068a8b-20f3-4786-9962-076c5327a598 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.135650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2068a8b-20f3-4786-9962-076c5327a598 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.137494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2068a8b-20f3-4786-9962-076c5327a598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.140117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2068a8b-20f3-4786-9962-076c5327a598 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.140604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 800/1601] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:29.141937 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c79a1-f930-47f7-bde7-d3893e17e466 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.142389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c79a1-f930-47f7-bde7-d3893e17e466 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.142677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c79a1-f930-47f7-bde7-d3893e17e466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.154396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 802/1602] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:29.155508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2068a8b-20f3-4786-9962-076c5327a598 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.166331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-9d1c79a1-f930-47f7-bde7-d3893e17e466 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:29.166562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2068a8b-20f3-4786-9962-076c5327a598 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.167734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9d1c79a1-f930-47f7-bde7-d3893e17e466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:29.171840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195ebd2-536c-4252-832d-6eacd9d95974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.172826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ebd2-536c-4252-832d-6eacd9d95974 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.173150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ebd2-536c-4252-832d-6eacd9d95974 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.173386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195ebd2-536c-4252-832d-6eacd9d95974 admin admin] PATH_INFO: `/v3/projects/a779b000010c4549989d8a8f25e8b1be/users/927ffbcd7078473f97f42bdb54acf57e/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.177657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 801/1603] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:29.178468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c79a1-f930-47f7-bde7-d3893e17e466 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.189947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c79a1-f930-47f7-bde7-d3893e17e466 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.194587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195ebd2-536c-4252-832d-6eacd9d95974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a779b000010c4549989d8a8f25e8b1be, user_id=927ffbcd7078473f97f42bdb54acf57e, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.196858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4275de-a99c-4170-b9b3-c9c76816cc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.197631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195ebd2-536c-4252-832d-6eacd9d95974 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.197915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4275de-a99c-4170-b9b3-c9c76816cc05 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.198477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4275de-a99c-4170-b9b3-c9c76816cc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.198954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4275de-a99c-4170-b9b3-c9c76816cc05 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.199845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4275de-a99c-4170-b9b3-c9c76816cc05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.201979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4275de-a99c-4170-b9b3-c9c76816cc05 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.215376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 803/1604] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:29 2022] PUT /identity/v3/projects/a779b000010c4549989d8a8f25e8b1be/users/927ffbcd7078473f97f42bdb54acf57e/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:29.219331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195ebd2-536c-4252-832d-6eacd9d95974 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.220263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 802/1605] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:29.221510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4275de-a99c-4170-b9b3-c9c76816cc05 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.234510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4275de-a99c-4170-b9b3-c9c76816cc05 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.256432 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:29.297359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195ebd2-536c-4252-832d-6eacd9d95974 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.303267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bf387-a5d5-4b7e-88a7-d5afc701eaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.304275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bf387-a5d5-4b7e-88a7-d5afc701eaad admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.304981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bf387-a5d5-4b7e-88a7-d5afc701eaad admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.304981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bf387-a5d5-4b7e-88a7-d5afc701eaad admin admin] PATH_INFO: `/v3/projects/bba3035f3a99457eb9502a492612f37e/users/63db9dc1952d4c0e8686a89cdf4c5fab/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.306709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27ec7b-f809-40eb-b034-7766cf7794ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.307684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27ec7b-f809-40eb-b034-7766cf7794ac admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.308051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27ec7b-f809-40eb-b034-7766cf7794ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.308349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27ec7b-f809-40eb-b034-7766cf7794ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.309360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27ec7b-f809-40eb-b034-7766cf7794ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.311746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27ec7b-f809-40eb-b034-7766cf7794ac admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.322818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bf387-a5d5-4b7e-88a7-d5afc701eaad admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba3035f3a99457eb9502a492612f37e, user_id=63db9dc1952d4c0e8686a89cdf4c5fab, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.326046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bf387-a5d5-4b7e-88a7-d5afc701eaad admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.327235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 803/1606] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:29.328094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27ec7b-f809-40eb-b034-7766cf7794ac admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.340264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27ec7b-f809-40eb-b034-7766cf7794ac admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.343279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 804/1607] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:29 2022] PUT /identity/v3/projects/bba3035f3a99457eb9502a492612f37e/users/63db9dc1952d4c0e8686a89cdf4c5fab/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:29.353538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bf387-a5d5-4b7e-88a7-d5afc701eaad admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.404159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0397c89-2909-4c08-a620-486e3cd1addd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.405378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0397c89-2909-4c08-a620-486e3cd1addd admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.405625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0397c89-2909-4c08-a620-486e3cd1addd admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.405971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0397c89-2909-4c08-a620-486e3cd1addd admin admin] PATH_INFO: `/v3/projects/33c3b742f5de48c9be3495113b017b2b/users/ef43a04070fa4955addbe514c661c974/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.421912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0397c89-2909-4c08-a620-486e3cd1addd admin admin] RBAC: Authorizing `identity:create_grant(project_id=33c3b742f5de48c9be3495113b017b2b, user_id=ef43a04070fa4955addbe514c661c974, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.425008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0397c89-2909-4c08-a620-486e3cd1addd admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.426556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bf387-a5d5-4b7e-88a7-d5afc701eaad admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.427537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:29.434766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae5770-4019-447e-a01a-d954e1bc9c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.436051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae5770-4019-447e-a01a-d954e1bc9c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.436430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae5770-4019-447e-a01a-d954e1bc9c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.436826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae5770-4019-447e-a01a-d954e1bc9c7f admin admin] PATH_INFO: `/v3/projects/7ee4f2aaeed54d0fb16474b971f333d6/users/5eb7e8b4befd4d519b440c632b59e724/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.445252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 804/1608] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:29 2022] PUT /identity/v3/projects/33c3b742f5de48c9be3495113b017b2b/users/ef43a04070fa4955addbe514c661c974/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:29.446385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0397c89-2909-4c08-a620-486e3cd1addd admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.464627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae5770-4019-447e-a01a-d954e1bc9c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee4f2aaeed54d0fb16474b971f333d6, user_id=5eb7e8b4befd4d519b440c632b59e724, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.467515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae5770-4019-447e-a01a-d954e1bc9c7f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.483319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 805/1609] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:29 2022] PUT /identity/v3/projects/7ee4f2aaeed54d0fb16474b971f333d6/users/5eb7e8b4befd4d519b440c632b59e724/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:29.484861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae5770-4019-447e-a01a-d954e1bc9c7f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.532540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0397c89-2909-4c08-a620-486e3cd1addd admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.597816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae5770-4019-447e-a01a-d954e1bc9c7f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.603216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee77937-054f-4a99-b6e6-9dec219d0a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.604267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee77937-054f-4a99-b6e6-9dec219d0a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.604696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee77937-054f-4a99-b6e6-9dec219d0a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.605042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee77937-054f-4a99-b6e6-9dec219d0a60 admin admin] PATH_INFO: `/v3/projects/040134c568aa46c09ca9e5b077cc5d3c/users/04e35f840ef045f0b44fab3e57a1a3c8/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.623378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a4bc9-dcc9-4a0d-8cbc-057a19588357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.624121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a4bc9-dcc9-4a0d-8cbc-057a19588357 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.625123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee77937-054f-4a99-b6e6-9dec219d0a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=040134c568aa46c09ca9e5b077cc5d3c, user_id=04e35f840ef045f0b44fab3e57a1a3c8, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.625591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a4bc9-dcc9-4a0d-8cbc-057a19588357 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.625889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a4bc9-dcc9-4a0d-8cbc-057a19588357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.628719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee77937-054f-4a99-b6e6-9dec219d0a60 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.640031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a4bc9-dcc9-4a0d-8cbc-057a19588357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.641795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a4bc9-dcc9-4a0d-8cbc-057a19588357 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.647794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 806/1610] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:29 2022] PUT /identity/v3/projects/040134c568aa46c09ca9e5b077cc5d3c/users/04e35f840ef045f0b44fab3e57a1a3c8/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:29.650064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfc16d-5c3f-481f-8406-717758ce78d1 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.650362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfc16d-5c3f-481f-8406-717758ce78d1 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.651154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfc16d-5c3f-481f-8406-717758ce78d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.655864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 805/1611] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:29.657055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a4bc9-dcc9-4a0d-8cbc-057a19588357 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:29.667488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a4bc9-dcc9-4a0d-8cbc-057a19588357 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:29.670651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-16bfc16d-5c3f-481f-8406-717758ce78d1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:29.671880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-16bfc16d-5c3f-481f-8406-717758ce78d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:29.687239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:29.753180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674db2f-f0f8-4ea2-b5a7-d339fa4458d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:29.754789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674db2f-f0f8-4ea2-b5a7-d339fa4458d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.755077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674db2f-f0f8-4ea2-b5a7-d339fa4458d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.755356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674db2f-f0f8-4ea2-b5a7-d339fa4458d0 admin admin] PATH_INFO: `/v3/projects/007fd89db2e04aee9e6962b7bf79e90e/users/1cdd7c6081a64443bef9da5aa173b06e/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.756677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 807/1612] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:29.758286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974049b-a4d7-4a0b-b1bf-584607b92a1f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.758578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974049b-a4d7-4a0b-b1bf-584607b92a1f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.758880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974049b-a4d7-4a0b-b1bf-584607b92a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.775750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674db2f-f0f8-4ea2-b5a7-d339fa4458d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=007fd89db2e04aee9e6962b7bf79e90e, user_id=1cdd7c6081a64443bef9da5aa173b06e, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:29.779038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674db2f-f0f8-4ea2-b5a7-d339fa4458d0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:29.792051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c974049b-a4d7-4a0b-b1bf-584607b92a1f None None] MFA Rules not processed for user `63db9dc1952d4c0e8686a89cdf4c5fab`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:29.793505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c974049b-a4d7-4a0b-b1bf-584607b92a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:29.800217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 806/1613] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:29 2022] PUT /identity/v3/projects/007fd89db2e04aee9e6962b7bf79e90e/users/1cdd7c6081a64443bef9da5aa173b06e/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:29.802641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907119b-6f7f-4ad8-b894-8e98e53d99b4 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.803302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907119b-6f7f-4ad8-b894-8e98e53d99b4 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.803758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907119b-6f7f-4ad8-b894-8e98e53d99b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.847869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-b907119b-6f7f-4ad8-b894-8e98e53d99b4 None None] MFA Rules not processed for user `ef43a04070fa4955addbe514c661c974`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:29.850232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b907119b-6f7f-4ad8-b894-8e98e53d99b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:29.887640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 808/1614] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:29.892622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:29.895883 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbca63-8d46-477f-8625-a086b8e086b4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.896382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbca63-8d46-477f-8625-a086b8e086b4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.897817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbca63-8d46-477f-8625-a086b8e086b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:29.924803 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a2bbca63-8d46-477f-8625-a086b8e086b4 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:29.927107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a2bbca63-8d46-477f-8625-a086b8e086b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:29.959775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 807/1615] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:29.962056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad0dcc-f655-4ffc-a544-3a3565420f01 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:29.962436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad0dcc-f655-4ffc-a544-3a3565420f01 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:29.962787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad0dcc-f655-4ffc-a544-3a3565420f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.008138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-2cad0dcc-f655-4ffc-a544-3a3565420f01 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:30.009482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2cad0dcc-f655-4ffc-a544-3a3565420f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:30.038801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 809/1616] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:30.041061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbca63-8d46-477f-8625-a086b8e086b4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:30.043368 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a2bbca63-8d46-477f-8625-a086b8e086b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:30.130640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 808/1617] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:30.131965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad0dcc-f655-4ffc-a544-3a3565420f01 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:30.132310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:30.139943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbca63-8d46-477f-8625-a086b8e086b4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:30.144675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad0dcc-f655-4ffc-a544-3a3565420f01 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:30.145454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318b3cd-808d-4999-9509-f2dd6728b863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:30.146341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318b3cd-808d-4999-9509-f2dd6728b863 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.146716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318b3cd-808d-4999-9509-f2dd6728b863 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.147253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318b3cd-808d-4999-9509-f2dd6728b863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.151092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af3b57-810c-46c6-8683-f355d72195f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:30.152747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af3b57-810c-46c6-8683-f355d72195f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.153185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af3b57-810c-46c6-8683-f355d72195f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.153733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af3b57-810c-46c6-8683-f355d72195f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.154814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af3b57-810c-46c6-8683-f355d72195f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:30.156662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af3b57-810c-46c6-8683-f355d72195f2 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:30.161694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318b3cd-808d-4999-9509-f2dd6728b863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:30.163273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318b3cd-808d-4999-9509-f2dd6728b863 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:30.173363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 809/1618] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:30.174572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af3b57-810c-46c6-8683-f355d72195f2 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:30.177601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 810/1619] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:30.179225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c7fd3-ad32-4051-a015-b208ac1810c2 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.179629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c7fd3-ad32-4051-a015-b208ac1810c2 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.180121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c7fd3-ad32-4051-a015-b208ac1810c2 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.181701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 811/1620] 10.222.0.47 () {54 vars in 961 bytes} [Fri Jun 24 23:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:48:30.182383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c7fd3-ad32-4051-a015-b208ac1810c2 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:30.196207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c7fd3-ad32-4051-a015-b208ac1810c2 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:30.202407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0b3c2-f666-45fd-8788-5cddb81ab96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:30.203205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0b3c2-f666-45fd-8788-5cddb81ab96d service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.203597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0b3c2-f666-45fd-8788-5cddb81ab96d service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.203968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0b3c2-f666-45fd-8788-5cddb81ab96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.221126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0b3c2-f666-45fd-8788-5cddb81ab96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:30.223650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0b3c2-f666-45fd-8788-5cddb81ab96d service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:30.240068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 812/1621] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:30.241675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0b3c2-f666-45fd-8788-5cddb81ab96d service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:30.254139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0b3c2-f666-45fd-8788-5cddb81ab96d service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:30.261335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ea3a-c085-4869-bc05-b0d2b7dbaac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:30.262073 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ea3a-c085-4869-bc05-b0d2b7dbaac9 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.262493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ea3a-c085-4869-bc05-b0d2b7dbaac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.263261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99ea3a-c085-4869-bc05-b0d2b7dbaac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.278467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99ea3a-c085-4869-bc05-b0d2b7dbaac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:30.280714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99ea3a-c085-4869-bc05-b0d2b7dbaac9 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:30.295091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 813/1622] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:30.296101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ea3a-c085-4869-bc05-b0d2b7dbaac9 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:30.307417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99ea3a-c085-4869-bc05-b0d2b7dbaac9 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:30.314064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a5a3c-115f-4eba-9596-bbefb6b20a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:30.315055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a5a3c-115f-4eba-9596-bbefb6b20a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.315401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a5a3c-115f-4eba-9596-bbefb6b20a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.316325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a5a3c-115f-4eba-9596-bbefb6b20a40 admin admin] PATH_INFO: `/v3/projects/007fd89db2e04aee9e6962b7bf79e90e/users/1cdd7c6081a64443bef9da5aa173b06e/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.318339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a5a3c-115f-4eba-9596-bbefb6b20a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=007fd89db2e04aee9e6962b7bf79e90e, user_id=1cdd7c6081a64443bef9da5aa173b06e, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:30.321664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a5a3c-115f-4eba-9596-bbefb6b20a40 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:30.324465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af3b57-810c-46c6-8683-f355d72195f2 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:30.334991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c79b95-e335-4254-b020-36081e3d0b23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:30.340697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 814/1623] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:30 2022] PUT /identity/v3/projects/007fd89db2e04aee9e6962b7bf79e90e/users/1cdd7c6081a64443bef9da5aa173b06e/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:30.341032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c79b95-e335-4254-b020-36081e3d0b23 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.341032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c79b95-e335-4254-b020-36081e3d0b23 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.341032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c79b95-e335-4254-b020-36081e3d0b23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.341248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a207a4-0bd1-487d-9c73-18cc7eb71547 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.341248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a207a4-0bd1-487d-9c73-18cc7eb71547 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.341248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a207a4-0bd1-487d-9c73-18cc7eb71547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.372650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c79b95-e335-4254-b020-36081e3d0b23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:30.374169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c79b95-e335-4254-b020-36081e3d0b23 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:30.392609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a7a207a4-0bd1-487d-9c73-18cc7eb71547 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:30.394250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a7a207a4-0bd1-487d-9c73-18cc7eb71547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:30.398971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 810/1624] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:30.400319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0abc5-2f8b-4d54-a358-3361882e2845 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.400623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0abc5-2f8b-4d54-a358-3361882e2845 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.400909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0abc5-2f8b-4d54-a358-3361882e2845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.474328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-73d0abc5-2f8b-4d54-a358-3361882e2845 None None] MFA Rules not processed for user `1cdd7c6081a64443bef9da5aa173b06e`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:30.475653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-73d0abc5-2f8b-4d54-a358-3361882e2845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:30.566657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 815/1625] 10.222.0.47 () {58 vars in 1058 bytes} [Fri Jun 24 23:48:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:30.596830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a207a4-0bd1-487d-9c73-18cc7eb71547 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:30.601247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:30.629281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a207a4-0bd1-487d-9c73-18cc7eb71547 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:30.641345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cb93e-8891-45c6-a7a9-323abe3b1605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:30.642190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cb93e-8891-45c6-a7a9-323abe3b1605 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.643021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cb93e-8891-45c6-a7a9-323abe3b1605 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.644178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cb93e-8891-45c6-a7a9-323abe3b1605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.654474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 811/1626] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:30.669918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0abc5-2f8b-4d54-a358-3361882e2845 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:30.771863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cb93e-8891-45c6-a7a9-323abe3b1605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:30.776248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cb93e-8891-45c6-a7a9-323abe3b1605 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:30.841409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 816/1627] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:48:30.849091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0abc5-2f8b-4d54-a358-3361882e2845 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:30.861484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbefaba-f232-45fd-956d-21de6c35eb6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:30.863811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbefaba-f232-45fd-956d-21de6c35eb6a service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:30.864469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbefaba-f232-45fd-956d-21de6c35eb6a service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:30.866039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbefaba-f232-45fd-956d-21de6c35eb6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:30.906305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbefaba-f232-45fd-956d-21de6c35eb6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:30.908257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbefaba-f232-45fd-956d-21de6c35eb6a service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:30.933059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 812/1628] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:31.244001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c198bc-8358-407a-97c3-c65fffbc7cbe None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:31.244622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c198bc-8358-407a-97c3-c65fffbc7cbe None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:31.244622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c198bc-8358-407a-97c3-c65fffbc7cbe None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:31.249559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 817/1629] 10.222.0.47 () {54 vars in 912 bytes} [Fri Jun 24 23:48:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:48:31.256885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaec83-7360-4f40-9de5-b9346e23dc55 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:31.256885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaec83-7360-4f40-9de5-b9346e23dc55 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:31.256885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaec83-7360-4f40-9de5-b9346e23dc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:31.280268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-68eaec83-7360-4f40-9de5-b9346e23dc55 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:31.282045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-68eaec83-7360-4f40-9de5-b9346e23dc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:31.293826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 813/1630] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:48:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:31.316470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c198bc-8358-407a-97c3-c65fffbc7cbe None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:31.339328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c198bc-8358-407a-97c3-c65fffbc7cbe None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:31.349239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0a23a-05b9-4497-b243-167aeab4e7d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:31.349905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0a23a-05b9-4497-b243-167aeab4e7d4 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:31.350191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0a23a-05b9-4497-b243-167aeab4e7d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:31.350470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0a23a-05b9-4497-b243-167aeab4e7d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:31.380235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0a23a-05b9-4497-b243-167aeab4e7d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:31.382728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0a23a-05b9-4497-b243-167aeab4e7d4 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:31.430563 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 818/1631] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:32.165865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eaec83-7360-4f40-9de5-b9346e23dc55 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:32.354012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eaec83-7360-4f40-9de5-b9346e23dc55 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:32.371121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:32.374413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:32.374833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:32.375152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] PATH_INFO: `/v3/users/25ed640040f94fc085bc565e3519f231` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:32.376669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ed640040f94fc085bc565e3519f231)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:32.377731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: WARNING py.warnings [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:48:32.377731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: warnings.warn(msg) Jun 24 23:48:32.377731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: Jun 24 23:48:32.379785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:32.486851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ed640040f94fc085bc565e3519f231'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:32.524863 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ed640040f94fc085bc565e3519f231'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:32.553020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ed640040f94fc085bc565e3519f231'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:32.553666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ed640040f94fc085bc565e3519f231'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:32.578134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ed640040f94fc085bc565e3519f231'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:32.580435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:32.599100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 814/1632] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:48:32 2022] DELETE /identity/v3/users/25ed640040f94fc085bc565e3519f231 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:32.601211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabaaef7-9b17-48a5-86b0-25e365f20974 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:32.601720 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabaaef7-9b17-48a5-86b0-25e365f20974 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:32.602048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabaaef7-9b17-48a5-86b0-25e365f20974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:32.637026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-eabaaef7-9b17-48a5-86b0-25e365f20974 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:32.638921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-eabaaef7-9b17-48a5-86b0-25e365f20974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:32.697372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:32.801903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 819/1633] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:32.826956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:32.829202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:32.968106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee5229-ffc6-4119-9253-456d91307f1f None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:32.968549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee5229-ffc6-4119-9253-456d91307f1f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:32.969017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee5229-ffc6-4119-9253-456d91307f1f None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:32.972208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 820/1634] 10.222.0.47 () {54 vars in 961 bytes} [Fri Jun 24 23:48:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:48:32.986459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775904a-0cde-4ff3-b20f-d9613f6376f9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:32.992004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c071ca5-318c-42b1-942c-077c5d0745fb None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:32.992576 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c071ca5-318c-42b1-942c-077c5d0745fb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:32.993183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c071ca5-318c-42b1-942c-077c5d0745fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:32.999398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab8dc8-eb84-4e38-bc2f-02e9c38d4105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:33.000112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab8dc8-eb84-4e38-bc2f-02e9c38d4105 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:33.000395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab8dc8-eb84-4e38-bc2f-02e9c38d4105 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:33.000712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab8dc8-eb84-4e38-bc2f-02e9c38d4105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:33.020107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab8dc8-eb84-4e38-bc2f-02e9c38d4105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:33.023622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab8dc8-eb84-4e38-bc2f-02e9c38d4105 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:33.042466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1c071ca5-318c-42b1-942c-077c5d0745fb None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:33.044743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1c071ca5-318c-42b1-942c-077c5d0745fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:33.056500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 815/1635] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:33.223397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 821/1636] 10.222.0.47 () {58 vars in 1058 bytes} [Fri Jun 24 23:48:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:48:33.225068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f6e40-dfe4-4dc4-9369-2de8cebd8a5a None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:33.225467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f6e40-dfe4-4dc4-9369-2de8cebd8a5a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:33.225874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f6e40-dfe4-4dc4-9369-2de8cebd8a5a None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:33.230342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 816/1637] 10.222.0.47 () {54 vars in 912 bytes} [Fri Jun 24 23:48:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:48:33.238826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb48335-61fe-45f1-96ae-301f275190d1 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:33.239130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb48335-61fe-45f1-96ae-301f275190d1 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:33.239714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb48335-61fe-45f1-96ae-301f275190d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:33.273371 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-8eb48335-61fe-45f1-96ae-301f275190d1 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:33.274801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8eb48335-61fe-45f1-96ae-301f275190d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:33.302571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 822/1638] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:48:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:33.319635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f6e40-dfe4-4dc4-9369-2de8cebd8a5a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:33.321528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a98f6e40-dfe4-4dc4-9369-2de8cebd8a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:33.460504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f6e40-dfe4-4dc4-9369-2de8cebd8a5a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:33.467758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7148ce-9462-4cd9-9ccc-e7c6944d0ac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:33.468878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7148ce-9462-4cd9-9ccc-e7c6944d0ac3 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:33.469460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7148ce-9462-4cd9-9ccc-e7c6944d0ac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:33.469879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7148ce-9462-4cd9-9ccc-e7c6944d0ac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:33.488136 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7148ce-9462-4cd9-9ccc-e7c6944d0ac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:33.490080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7148ce-9462-4cd9-9ccc-e7c6944d0ac3 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:33.505393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 817/1639] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:35.402230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb48335-61fe-45f1-96ae-301f275190d1 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:35.405744 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8eb48335-61fe-45f1-96ae-301f275190d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:35.440073 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb48335-61fe-45f1-96ae-301f275190d1 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:35.447595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:35.449541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:35.450749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:35.451231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] PATH_INFO: `/v3/projects/55c71ddb236848a387dd9949b44d4bd2` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:35.454064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c71ddb236848a387dd9949b44d4bd2)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:35.455487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: WARNING py.warnings [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 23:48:35.455487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: warnings.warn(msg) Jun 24 23:48:35.455487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: Jun 24 23:48:35.458110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:35.564083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c71ddb236848a387dd9949b44d4bd2', 'request_id': 'req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:35.586186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c71ddb236848a387dd9949b44d4bd2', 'request_id': 'req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:35.619134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c71ddb236848a387dd9949b44d4bd2', 'request_id': 'req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:35.646044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] The token cache is being invalidate because project 55c71ddb236848a387dd9949b44d4bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:48:35.646374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:35.648731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:35.661003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7148ce-9462-4cd9-9ccc-e7c6944d0ac3 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:35.661667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 823/1640] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:48:35 2022] DELETE /identity/v3/projects/55c71ddb236848a387dd9949b44d4bd2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:35.675911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7148ce-9462-4cd9-9ccc-e7c6944d0ac3 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:35.693345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:35.693345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:35.784837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:35.830048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:35.831346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:35.831840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:35.832343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] PATH_INFO: `/v3/users/1cdd7c6081a64443bef9da5aa173b06e` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:35.834833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cdd7c6081a64443bef9da5aa173b06e)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:35.839152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:35.844430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e501e-82db-4263-b76c-6ed3fcf4ecfb admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:35.860938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:35.861855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:35.862145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:35.862426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] PATH_INFO: `/v3/users/ef43a04070fa4955addbe514c661c974` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:35.863721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef43a04070fa4955addbe514c661c974)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:35.877522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:35.980709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:35.983990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cdd7c6081a64443bef9da5aa173b06e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.020636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cdd7c6081a64443bef9da5aa173b06e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.025850 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef43a04070fa4955addbe514c661c974'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.041947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cdd7c6081a64443bef9da5aa173b06e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.042663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cdd7c6081a64443bef9da5aa173b06e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.059044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-06b9f214-1a18-4647-9c60-2c6c6f54cf1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cdd7c6081a64443bef9da5aa173b06e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.070512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef43a04070fa4955addbe514c661c974'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.072368 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef43a04070fa4955addbe514c661c974'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.073420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef43a04070fa4955addbe514c661c974'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.078715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 818/1641] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:48:35 2022] DELETE /identity/v3/users/1cdd7c6081a64443bef9da5aa173b06e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:36.101541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef43a04070fa4955addbe514c661c974'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:36.138495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 824/1642] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:48:35 2022] DELETE /identity/v3/users/ef43a04070fa4955addbe514c661c974 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:36.143827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfd1c5-436d-44d4-8bfd-ad3ebf05fba8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:36.144318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfd1c5-436d-44d4-8bfd-ad3ebf05fba8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:36.144858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfd1c5-436d-44d4-8bfd-ad3ebf05fba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:36.169397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-20dfd1c5-436d-44d4-8bfd-ad3ebf05fba8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:36.171279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-20dfd1c5-436d-44d4-8bfd-ad3ebf05fba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:36.369691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 819/1643] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:48:36.390562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:36.392550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:36.526839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed780c7-bb53-4265-b254-a2db67219810 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:36.533792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a48668-2a64-409c-824a-66c36d34f6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:36.551939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a48668-2a64-409c-824a-66c36d34f6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:36.552432 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a48668-2a64-409c-824a-66c36d34f6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:36.552924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a48668-2a64-409c-824a-66c36d34f6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:36.579484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a48668-2a64-409c-824a-66c36d34f6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:36.582232 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a48668-2a64-409c-824a-66c36d34f6f2 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:36.601116 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 825/1644] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:37.346859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfd1c5-436d-44d4-8bfd-ad3ebf05fba8 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:37.353676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-20dfd1c5-436d-44d4-8bfd-ad3ebf05fba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:37.390682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfd1c5-436d-44d4-8bfd-ad3ebf05fba8 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:37.408058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:37.436326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:37.436769 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:37.437066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] PATH_INFO: `/v3/projects/007fd89db2e04aee9e6962b7bf79e90e` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:37.440951 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=007fd89db2e04aee9e6962b7bf79e90e)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:37.460763 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:37.664421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '007fd89db2e04aee9e6962b7bf79e90e', 'request_id': 'req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:37.687790 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '007fd89db2e04aee9e6962b7bf79e90e', 'request_id': 'req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:37.730788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '007fd89db2e04aee9e6962b7bf79e90e', 'request_id': 'req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:37.773647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] The token cache is being invalidate because project 007fd89db2e04aee9e6962b7bf79e90e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:48:37.774399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:37.776803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-28e5d6a9-0bbc-4b7b-8dc7-630983020d55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:37.825887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1438a-19bf-438e-a3a7-8ad8f0c7d1bd None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:37.827535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1438a-19bf-438e-a3a7-8ad8f0c7d1bd None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:37.828465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1438a-19bf-438e-a3a7-8ad8f0c7d1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:37.830935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 820/1645] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:48:37 2022] DELETE /identity/v3/projects/007fd89db2e04aee9e6962b7bf79e90e => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:37.864514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-dea1438a-19bf-438e-a3a7-8ad8f0c7d1bd None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:37.866771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-dea1438a-19bf-438e-a3a7-8ad8f0c7d1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:37.983628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:38.008858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 826/1646] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:38.018861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d638b44-c156-4750-a38d-146c043508a8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.019156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d638b44-c156-4750-a38d-146c043508a8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.019438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d638b44-c156-4750-a38d-146c043508a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.053484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0d638b44-c156-4750-a38d-146c043508a8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:38.054518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0d638b44-c156-4750-a38d-146c043508a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:38.075860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 821/1647] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:38.077096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1438a-19bf-438e-a3a7-8ad8f0c7d1bd None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:38.100689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1438a-19bf-438e-a3a7-8ad8f0c7d1bd None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:38.113058 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096be52-a04b-4a62-8332-1429868c382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:38.113819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096be52-a04b-4a62-8332-1429868c382e admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.114101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096be52-a04b-4a62-8332-1429868c382e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.114375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096be52-a04b-4a62-8332-1429868c382e admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.115503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096be52-a04b-4a62-8332-1429868c382e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:38.115822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096be52-a04b-4a62-8332-1429868c382e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:38.116730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d638b44-c156-4750-a38d-146c043508a8 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:38.117568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096be52-a04b-4a62-8332-1429868c382e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:38.118608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0d638b44-c156-4750-a38d-146c043508a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:38.143077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d638b44-c156-4750-a38d-146c043508a8 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:38.149666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 827/1648] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:38.152209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096be52-a04b-4a62-8332-1429868c382e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:38.154260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:38.155344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.155775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.156207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] PATH_INFO: `/v3/projects/33c3b742f5de48c9be3495113b017b2b` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.165165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=33c3b742f5de48c9be3495113b017b2b)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:38.166401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096be52-a04b-4a62-8332-1429868c382e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:38.167529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:38.197335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49bc75-dc61-420c-bda1-506d66982d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:38.199009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49bc75-dc61-420c-bda1-506d66982d6d admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.200868 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49bc75-dc61-420c-bda1-506d66982d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.202741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49bc75-dc61-420c-bda1-506d66982d6d admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.206981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49bc75-dc61-420c-bda1-506d66982d6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:38.220094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49bc75-dc61-420c-bda1-506d66982d6d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:38.289382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:38.346930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 828/1649] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:38 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:38.352084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49bc75-dc61-420c-bda1-506d66982d6d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:38.356085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33c3b742f5de48c9be3495113b017b2b', 'request_id': 'req-f8fb2760-3143-4fdc-bc15-37b52d6048fe'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:38.379311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33c3b742f5de48c9be3495113b017b2b', 'request_id': 'req-f8fb2760-3143-4fdc-bc15-37b52d6048fe'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:38.405629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33c3b742f5de48c9be3495113b017b2b', 'request_id': 'req-f8fb2760-3143-4fdc-bc15-37b52d6048fe'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:38.433315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] The token cache is being invalidate because project 33c3b742f5de48c9be3495113b017b2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:48:38.433631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:38.434584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f8fb2760-3143-4fdc-bc15-37b52d6048fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:38.436651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 822/1650] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:48:38 2022] DELETE /identity/v3/projects/33c3b742f5de48c9be3495113b017b2b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:48:38.443378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e9c2-8a42-45cc-a003-957342b40fc6 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.445192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e9c2-8a42-45cc-a003-957342b40fc6 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.445585 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e9c2-8a42-45cc-a003-957342b40fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.517994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-4fe5e9c2-8a42-45cc-a003-957342b40fc6 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:38.519467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4fe5e9c2-8a42-45cc-a003-957342b40fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:38.560295 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49bc75-dc61-420c-bda1-506d66982d6d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:38.583335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfa376-56f5-4e86-be46-2aefa4c30e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:38.583556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfa376-56f5-4e86-be46-2aefa4c30e8d admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.583829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfa376-56f5-4e86-be46-2aefa4c30e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.584112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfa376-56f5-4e86-be46-2aefa4c30e8d admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.585215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfa376-56f5-4e86-be46-2aefa4c30e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:38.586840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfa376-56f5-4e86-be46-2aefa4c30e8d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:38.654932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 829/1651] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:38 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:38.656256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfa376-56f5-4e86-be46-2aefa4c30e8d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:38.664075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-51dfa376-56f5-4e86-be46-2aefa4c30e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:38.705614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfa376-56f5-4e86-be46-2aefa4c30e8d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:38.716454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090b74a-68ef-4a07-aab9-6cc66427c8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:38.717164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090b74a-68ef-4a07-aab9-6cc66427c8bb admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.717967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090b74a-68ef-4a07-aab9-6cc66427c8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.717967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090b74a-68ef-4a07-aab9-6cc66427c8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.718869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090b74a-68ef-4a07-aab9-6cc66427c8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:38.720619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090b74a-68ef-4a07-aab9-6cc66427c8bb admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:38.725897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 823/1652] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:38.736324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75095e36-8033-41d1-99b4-d514eaa46364 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.737088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75095e36-8033-41d1-99b4-d514eaa46364 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.737445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75095e36-8033-41d1-99b4-d514eaa46364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.767598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 830/1653] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:48:38.774626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090b74a-68ef-4a07-aab9-6cc66427c8bb admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:38.818826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090b74a-68ef-4a07-aab9-6cc66427c8bb admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:38.838131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-75095e36-8033-41d1-99b4-d514eaa46364 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:38.842903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-75095e36-8033-41d1-99b4-d514eaa46364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:38.846203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0d1a6-5f16-4bff-8da9-c601142e15f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:38.846834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0d1a6-5f16-4bff-8da9-c601142e15f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.847121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0d1a6-5f16-4bff-8da9-c601142e15f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.847406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0d1a6-5f16-4bff-8da9-c601142e15f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.848148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0d1a6-5f16-4bff-8da9-c601142e15f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:38.849917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0d1a6-5f16-4bff-8da9-c601142e15f2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:38.891791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 824/1654] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:38.905018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75095e36-8033-41d1-99b4-d514eaa46364 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:38.907094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 831/1655] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:38.907620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0d1a6-5f16-4bff-8da9-c601142e15f2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:38.939645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75095e36-8033-41d1-99b4-d514eaa46364 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:38.955558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0d1a6-5f16-4bff-8da9-c601142e15f2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:38.963950 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca066974-1091-4f94-8a65-cdda87c2a4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:38.966578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca066974-1091-4f94-8a65-cdda87c2a4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.970249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca066974-1091-4f94-8a65-cdda87c2a4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.970909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca066974-1091-4f94-8a65-cdda87c2a4b8 admin admin] PATH_INFO: `/v3/projects/b9f2b9eddebb4db68270964f0ba1b826/users/636885900e904df38bd2020a3b57d3d8/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.976271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982726b-02f2-4331-8502-b15cd693e54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:38.977158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982726b-02f2-4331-8502-b15cd693e54c admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:38.977442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982726b-02f2-4331-8502-b15cd693e54c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:38.977719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982726b-02f2-4331-8502-b15cd693e54c admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:38.978459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982726b-02f2-4331-8502-b15cd693e54c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:38.978766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982726b-02f2-4331-8502-b15cd693e54c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:38.980411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982726b-02f2-4331-8502-b15cd693e54c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:39.045895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca066974-1091-4f94-8a65-cdda87c2a4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f2b9eddebb4db68270964f0ba1b826, user_id=636885900e904df38bd2020a3b57d3d8, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:39.048783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca066974-1091-4f94-8a65-cdda87c2a4b8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:39.073182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 832/1656] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:39.074465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982726b-02f2-4331-8502-b15cd693e54c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:39.105034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 825/1657] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:38 2022] PUT /identity/v3/projects/b9f2b9eddebb4db68270964f0ba1b826/users/636885900e904df38bd2020a3b57d3d8/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:39.108409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982726b-02f2-4331-8502-b15cd693e54c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:39.108931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a6431-2803-4c40-8e01-e5cf33f61f1d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:39.109230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a6431-2803-4c40-8e01-e5cf33f61f1d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:39.109519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a6431-2803-4c40-8e01-e5cf33f61f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:39.173641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d52a6431-2803-4c40-8e01-e5cf33f61f1d None None] MFA Rules not processed for user `636885900e904df38bd2020a3b57d3d8`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:39.179493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d52a6431-2803-4c40-8e01-e5cf33f61f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:39.282775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb4ed2-6378-44cf-a94b-42801969eb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:39.284514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb4ed2-6378-44cf-a94b-42801969eb7e admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:39.292800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb4ed2-6378-44cf-a94b-42801969eb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:39.294053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb4ed2-6378-44cf-a94b-42801969eb7e admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:39.295442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb4ed2-6378-44cf-a94b-42801969eb7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:39.301364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb4ed2-6378-44cf-a94b-42801969eb7e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:39.345917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 826/1658] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:39.347037 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a6431-2803-4c40-8e01-e5cf33f61f1d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:39.373574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a6431-2803-4c40-8e01-e5cf33f61f1d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:39.394563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1debdd1-00da-40c3-a11d-67cde6636a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:39.396050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1debdd1-00da-40c3-a11d-67cde6636a7b admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:39.396350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1debdd1-00da-40c3-a11d-67cde6636a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:39.396831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1debdd1-00da-40c3-a11d-67cde6636a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:39.398787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1debdd1-00da-40c3-a11d-67cde6636a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:39.402776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1debdd1-00da-40c3-a11d-67cde6636a7b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:39.425126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 833/1659] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:39 2022] POST /identity/v3/projects => generated 355 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:39.426945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb4ed2-6378-44cf-a94b-42801969eb7e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:39.438497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:39.456432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:39.473026 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 827/1660] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:39 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:39.476538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1debdd1-00da-40c3-a11d-67cde6636a7b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:39.636272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb4ed2-6378-44cf-a94b-42801969eb7e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:39.764331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1debdd1-00da-40c3-a11d-67cde6636a7b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:39.786172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b662401-e8e1-4835-8726-ce3faeedd6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:39.787153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b662401-e8e1-4835-8726-ce3faeedd6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:39.787763 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b662401-e8e1-4835-8726-ce3faeedd6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:39.788190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b662401-e8e1-4835-8726-ce3faeedd6d0 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:39.789077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b662401-e8e1-4835-8726-ce3faeedd6d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:39.791766 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b662401-e8e1-4835-8726-ce3faeedd6d0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:39.843739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125d23e-1e86-440d-8ca6-0f824c127ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:39.855170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125d23e-1e86-440d-8ca6-0f824c127ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:39.855928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125d23e-1e86-440d-8ca6-0f824c127ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:39.855928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125d23e-1e86-440d-8ca6-0f824c127ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:39.856549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125d23e-1e86-440d-8ca6-0f824c127ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:39.857046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 828/1661] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:39 2022] POST /identity/v3/users => generated 290 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:39.860563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b662401-e8e1-4835-8726-ce3faeedd6d0 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:39.863333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125d23e-1e86-440d-8ca6-0f824c127ec8 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:39.900402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b662401-e8e1-4835-8726-ce3faeedd6d0 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:39.909363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe565f7-1674-4492-a4e2-0e0e75b0dd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:39.910559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe565f7-1674-4492-a4e2-0e0e75b0dd0d admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:39.911080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe565f7-1674-4492-a4e2-0e0e75b0dd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:39.911709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe565f7-1674-4492-a4e2-0e0e75b0dd0d admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:39.912841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe565f7-1674-4492-a4e2-0e0e75b0dd0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:39.915592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe565f7-1674-4492-a4e2-0e0e75b0dd0d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:39.936404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 829/1662] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:39.937397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe565f7-1674-4492-a4e2-0e0e75b0dd0d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:39.944339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 834/1663] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:39 2022] POST /identity/v3/users => generated 278 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:39.945586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125d23e-1e86-440d-8ca6-0f824c127ec8 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:39.956196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe565f7-1674-4492-a4e2-0e0e75b0dd0d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:39.980639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125d23e-1e86-440d-8ca6-0f824c127ec8 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:39.984189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead68c2e-01a8-4156-830b-4e8c2014d487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:39.985441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead68c2e-01a8-4156-830b-4e8c2014d487 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:39.985949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead68c2e-01a8-4156-830b-4e8c2014d487 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:39.986596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead68c2e-01a8-4156-830b-4e8c2014d487 admin admin] PATH_INFO: `/v3/projects/9985ebd0b5464d34bb07758f80bd180a/users/933df08a4d3b471aafe9f0826133e6ac/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:39.991644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff634099-9165-4e51-b86f-e5cd2f8dd548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:39.992312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634099-9165-4e51-b86f-e5cd2f8dd548 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:39.992615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634099-9165-4e51-b86f-e5cd2f8dd548 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:39.992923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff634099-9165-4e51-b86f-e5cd2f8dd548 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:39.993756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff634099-9165-4e51-b86f-e5cd2f8dd548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:39.996971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff634099-9165-4e51-b86f-e5cd2f8dd548 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:40.007391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead68c2e-01a8-4156-830b-4e8c2014d487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9985ebd0b5464d34bb07758f80bd180a, user_id=933df08a4d3b471aafe9f0826133e6ac, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:40.020132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead68c2e-01a8-4156-830b-4e8c2014d487 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:40.032411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 835/1664] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:40.033666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff634099-9165-4e51-b86f-e5cd2f8dd548 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:40.047838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff634099-9165-4e51-b86f-e5cd2f8dd548 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:40.066051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 830/1665] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:39 2022] PUT /identity/v3/projects/9985ebd0b5464d34bb07758f80bd180a/users/933df08a4d3b471aafe9f0826133e6ac/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:40.067451 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead68c2e-01a8-4156-830b-4e8c2014d487 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:40.211347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc44f2e-ba2a-4cc9-8735-57bbc3dccc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:40.212057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc44f2e-ba2a-4cc9-8735-57bbc3dccc21 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:40.212347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc44f2e-ba2a-4cc9-8735-57bbc3dccc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:40.214814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc44f2e-ba2a-4cc9-8735-57bbc3dccc21 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:40.216160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc44f2e-ba2a-4cc9-8735-57bbc3dccc21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:40.219781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc44f2e-ba2a-4cc9-8735-57bbc3dccc21 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:40.231596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead68c2e-01a8-4156-830b-4e8c2014d487 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:40.244246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4a4fc-65e3-42ee-a758-97de02f5599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:40.244972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4a4fc-65e3-42ee-a758-97de02f5599f admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:40.245261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4a4fc-65e3-42ee-a758-97de02f5599f admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:40.245553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4a4fc-65e3-42ee-a758-97de02f5599f admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:40.247287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4a4fc-65e3-42ee-a758-97de02f5599f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:40.250951 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 836/1666] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:40.251505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4a4fc-65e3-42ee-a758-97de02f5599f admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:40.255104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc44f2e-ba2a-4cc9-8735-57bbc3dccc21 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:40.272436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc44f2e-ba2a-4cc9-8735-57bbc3dccc21 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:40.276216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 831/1667] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:40.282431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4a4fc-65e3-42ee-a758-97de02f5599f admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:40.291657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249623d6-0b64-49e1-a78e-6fa9ed95ac36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:40.292570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249623d6-0b64-49e1-a78e-6fa9ed95ac36 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:40.292890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249623d6-0b64-49e1-a78e-6fa9ed95ac36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:40.293176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249623d6-0b64-49e1-a78e-6fa9ed95ac36 admin admin] PATH_INFO: `/v3/projects/cee9fb8005f140feb75154f368f19b90/users/a4f97eb8e50e479fbf8403e2096b2867/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:40.302386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4a4fc-65e3-42ee-a758-97de02f5599f admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:40.315700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931fdd9-3ca8-47da-be8f-72308fd79b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:40.316534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931fdd9-3ca8-47da-be8f-72308fd79b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:40.317286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931fdd9-3ca8-47da-be8f-72308fd79b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:40.317286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931fdd9-3ca8-47da-be8f-72308fd79b15 admin admin] PATH_INFO: `/v3/projects/9985ebd0b5464d34bb07758f80bd180a/users/933df08a4d3b471aafe9f0826133e6ac/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:40.320855 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931fdd9-3ca8-47da-be8f-72308fd79b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9985ebd0b5464d34bb07758f80bd180a, user_id=933df08a4d3b471aafe9f0826133e6ac, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:40.323534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931fdd9-3ca8-47da-be8f-72308fd79b15 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:40.329160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249623d6-0b64-49e1-a78e-6fa9ed95ac36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee9fb8005f140feb75154f368f19b90, user_id=a4f97eb8e50e479fbf8403e2096b2867, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:40.332162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249623d6-0b64-49e1-a78e-6fa9ed95ac36 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:40.364132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 832/1668] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:40 2022] PUT /identity/v3/projects/9985ebd0b5464d34bb07758f80bd180a/users/933df08a4d3b471aafe9f0826133e6ac/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:40.377149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53801cf-edb8-414a-a07b-3558086c4668 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:40.377696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53801cf-edb8-414a-a07b-3558086c4668 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:40.377696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53801cf-edb8-414a-a07b-3558086c4668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:40.382606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 837/1669] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:40 2022] PUT /identity/v3/projects/cee9fb8005f140feb75154f368f19b90/users/a4f97eb8e50e479fbf8403e2096b2867/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:40.392748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b4163-f0ce-4947-889e-288a5365fff4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:40.393579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b4163-f0ce-4947-889e-288a5365fff4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:40.393579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b4163-f0ce-4947-889e-288a5365fff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:40.400264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:40.467118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a53801cf-edb8-414a-a07b-3558086c4668 None None] MFA Rules not processed for user `933df08a4d3b471aafe9f0826133e6ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:40.469456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a22b4163-f0ce-4947-889e-288a5365fff4 None None] MFA Rules not processed for user `a4f97eb8e50e479fbf8403e2096b2867`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:40.470681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a22b4163-f0ce-4947-889e-288a5365fff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:40.472812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a53801cf-edb8-414a-a07b-3558086c4668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:40.566321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:40.697971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 838/1670] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:40 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:40.699135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b4163-f0ce-4947-889e-288a5365fff4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:40.710124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 833/1671] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:40 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:40.719993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53801cf-edb8-414a-a07b-3558086c4668 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:40.721626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a53801cf-edb8-414a-a07b-3558086c4668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:40.890472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b4163-f0ce-4947-889e-288a5365fff4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:40.899163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60445a-46e4-4ca2-812f-ced779648436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:40.899876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53801cf-edb8-414a-a07b-3558086c4668 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:40.900234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60445a-46e4-4ca2-812f-ced779648436 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:40.900938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60445a-46e4-4ca2-812f-ced779648436 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:40.901381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60445a-46e4-4ca2-812f-ced779648436 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:40.902374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60445a-46e4-4ca2-812f-ced779648436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:40.904773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60445a-46e4-4ca2-812f-ced779648436 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:40.910568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c45a0-c475-42ea-bce5-c667e1b5c288 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:40.912120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c45a0-c475-42ea-bce5-c667e1b5c288 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:40.912472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c45a0-c475-42ea-bce5-c667e1b5c288 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:40.913116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c45a0-c475-42ea-bce5-c667e1b5c288 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:40.933455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c45a0-c475-42ea-bce5-c667e1b5c288 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:40.935701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c45a0-c475-42ea-bce5-c667e1b5c288 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:40.951825 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 834/1672] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:40.974087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 839/1673] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:40 2022] POST /identity/v3/projects => generated 357 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:40.978274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c45a0-c475-42ea-bce5-c667e1b5c288 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:40.999497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c45a0-c475-42ea-bce5-c667e1b5c288 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:41.341035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd41c3-8f94-40a6-a111-2580612465b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:41.348476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd41c3-8f94-40a6-a111-2580612465b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:41.348955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd41c3-8f94-40a6-a111-2580612465b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:41.349248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd41c3-8f94-40a6-a111-2580612465b2 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:41.350016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd41c3-8f94-40a6-a111-2580612465b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:41.352016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd41c3-8f94-40a6-a111-2580612465b2 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:41.429874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 835/1674] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:40 2022] POST /identity/v3/users => generated 285 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:41.433844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60445a-46e4-4ca2-812f-ced779648436 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:41.453183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60445a-46e4-4ca2-812f-ced779648436 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:41.480084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db0479-c613-42eb-8856-5d52d24e0135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:41.480988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db0479-c613-42eb-8856-5d52d24e0135 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:41.481408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db0479-c613-42eb-8856-5d52d24e0135 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:41.481697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db0479-c613-42eb-8856-5d52d24e0135 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:41.483811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db0479-c613-42eb-8856-5d52d24e0135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:41.501936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db0479-c613-42eb-8856-5d52d24e0135 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:41.547391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd41c3-8f94-40a6-a111-2580612465b2 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:41.560632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 840/1675] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:41.572988 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd41c3-8f94-40a6-a111-2580612465b2 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:41.599529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2154291-aca0-4f13-82fa-2df8015d6fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:41.600092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2154291-aca0-4f13-82fa-2df8015d6fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:41.600540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2154291-aca0-4f13-82fa-2df8015d6fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:41.601006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2154291-aca0-4f13-82fa-2df8015d6fdd admin admin] PATH_INFO: `/v3/projects/69b0880e11ec44cabce1e7f2dde601ba/users/ad79487e8f77428a80613d9847e8f1cd/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:41.651499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2154291-aca0-4f13-82fa-2df8015d6fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b0880e11ec44cabce1e7f2dde601ba, user_id=ad79487e8f77428a80613d9847e8f1cd, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:41.654533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2154291-aca0-4f13-82fa-2df8015d6fdd admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:41.702070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 836/1676] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:41 2022] PUT /identity/v3/projects/69b0880e11ec44cabce1e7f2dde601ba/users/ad79487e8f77428a80613d9847e8f1cd/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:41.705788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db0479-c613-42eb-8856-5d52d24e0135 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:41.723228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db0479-c613-42eb-8856-5d52d24e0135 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:41.839493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f49fc0-bd97-4367-b677-c2112a891336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:41.840929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f49fc0-bd97-4367-b677-c2112a891336 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:41.840929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f49fc0-bd97-4367-b677-c2112a891336 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:41.841722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f49fc0-bd97-4367-b677-c2112a891336 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:41.842134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f49fc0-bd97-4367-b677-c2112a891336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:41.844546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f49fc0-bd97-4367-b677-c2112a891336 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:41.868157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 841/1677] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:41 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:41.873261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2154291-aca0-4f13-82fa-2df8015d6fdd admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:41.898257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2154291-aca0-4f13-82fa-2df8015d6fdd admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:41.920877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56003f97-bdf2-4329-b055-5c388b05c31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:41.921798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56003f97-bdf2-4329-b055-5c388b05c31e admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:41.922079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56003f97-bdf2-4329-b055-5c388b05c31e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:41.922521 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56003f97-bdf2-4329-b055-5c388b05c31e admin admin] PATH_INFO: `/v3/projects/69b0880e11ec44cabce1e7f2dde601ba/users/ad79487e8f77428a80613d9847e8f1cd/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:41.928494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56003f97-bdf2-4329-b055-5c388b05c31e admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b0880e11ec44cabce1e7f2dde601ba, user_id=ad79487e8f77428a80613d9847e8f1cd, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:41.939684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56003f97-bdf2-4329-b055-5c388b05c31e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:41.972399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 837/1678] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:41 2022] PUT /identity/v3/projects/69b0880e11ec44cabce1e7f2dde601ba/users/ad79487e8f77428a80613d9847e8f1cd/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:48:41.984174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09d3e5-e006-4481-869b-1fb2e77f5c1b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:41.987190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09d3e5-e006-4481-869b-1fb2e77f5c1b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:41.987644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09d3e5-e006-4481-869b-1fb2e77f5c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:41.994914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:42.061371 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-fd09d3e5-e006-4481-869b-1fb2e77f5c1b None None] MFA Rules not processed for user `ad79487e8f77428a80613d9847e8f1cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:42.064036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-fd09d3e5-e006-4481-869b-1fb2e77f5c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:42.284185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 842/1679] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:41 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:42.291586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56003f97-bdf2-4329-b055-5c388b05c31e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:42.293637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-56003f97-bdf2-4329-b055-5c388b05c31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:42.392789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:42.502251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56003f97-bdf2-4329-b055-5c388b05c31e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:42.514880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dcda6-9c9d-46ca-b7da-293b8956cefc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:42.515723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dcda6-9c9d-46ca-b7da-293b8956cefc service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:42.516115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dcda6-9c9d-46ca-b7da-293b8956cefc service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:42.516890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dcda6-9c9d-46ca-b7da-293b8956cefc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:42.545663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99dcda6-9c9d-46ca-b7da-293b8956cefc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:42.552001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99dcda6-9c9d-46ca-b7da-293b8956cefc service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:42.580300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 838/1680] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:42 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:42.742414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09d3e5-e006-4481-869b-1fb2e77f5c1b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:42.755935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09d3e5-e006-4481-869b-1fb2e77f5c1b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:42.765008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777459f-c216-40dc-9a72-d3f94cc9a290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:42.765773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777459f-c216-40dc-9a72-d3f94cc9a290 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:42.766080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777459f-c216-40dc-9a72-d3f94cc9a290 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:42.766375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777459f-c216-40dc-9a72-d3f94cc9a290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:42.922500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777459f-c216-40dc-9a72-d3f94cc9a290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:42.924326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777459f-c216-40dc-9a72-d3f94cc9a290 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:42.944518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 843/1681] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:42 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:43.745850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dcda6-9c9d-46ca-b7da-293b8956cefc service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:43.747497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e99dcda6-9c9d-46ca-b7da-293b8956cefc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:43.875943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dcda6-9c9d-46ca-b7da-293b8956cefc service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:43.882629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:43.884660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:43.885160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:43.885641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] PATH_INFO: `/v3/users/63db9dc1952d4c0e8686a89cdf4c5fab` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:43.887662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=63db9dc1952d4c0e8686a89cdf4c5fab)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:43.889982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:43.976366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63db9dc1952d4c0e8686a89cdf4c5fab'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:43.993898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63db9dc1952d4c0e8686a89cdf4c5fab'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:44.012498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63db9dc1952d4c0e8686a89cdf4c5fab'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:44.013955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63db9dc1952d4c0e8686a89cdf4c5fab'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:44.028805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63db9dc1952d4c0e8686a89cdf4c5fab'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:44.040305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 839/1682] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:48:43 2022] DELETE /identity/v3/users/63db9dc1952d4c0e8686a89cdf4c5fab => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:44.046247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78213d98-48b4-4c53-be30-e0b9951f08f2 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:44.047264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78213d98-48b4-4c53-be30-e0b9951f08f2 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:44.047948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78213d98-48b4-4c53-be30-e0b9951f08f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:44.080639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-78213d98-48b4-4c53-be30-e0b9951f08f2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:44.082045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-78213d98-48b4-4c53-be30-e0b9951f08f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:44.266352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 844/1683] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:48:44.315158 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:44.316414 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:44.330501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78213d98-48b4-4c53-be30-e0b9951f08f2 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:44.334484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-78213d98-48b4-4c53-be30-e0b9951f08f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:44.357982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78213d98-48b4-4c53-be30-e0b9951f08f2 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:44.371091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:44.372340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:44.372810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:44.373247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] PATH_INFO: `/v3/users/636885900e904df38bd2020a3b57d3d8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:44.377180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=636885900e904df38bd2020a3b57d3d8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:44.379499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:44.429741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:44.458483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d99f1c-e6b3-471e-85a3-6d4be0539efe admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:44.463254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083b9a3-18dd-462f-8909-72c5c3a08dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:44.464064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b9a3-18dd-462f-8909-72c5c3a08dcc service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:44.464300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b9a3-18dd-462f-8909-72c5c3a08dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:44.464604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b9a3-18dd-462f-8909-72c5c3a08dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:44.477440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:44.483093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083b9a3-18dd-462f-8909-72c5c3a08dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:44.486235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083b9a3-18dd-462f-8909-72c5c3a08dcc service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:44.511743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 840/1684] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:44.515516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1487d-3ef0-4304-a6e1-1f1b9b88f56d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:44.515812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1487d-3ef0-4304-a6e1-1f1b9b88f56d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:44.516095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1487d-3ef0-4304-a6e1-1f1b9b88f56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:44.518969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '636885900e904df38bd2020a3b57d3d8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:44.536953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '636885900e904df38bd2020a3b57d3d8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:44.537789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '636885900e904df38bd2020a3b57d3d8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:44.538718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '636885900e904df38bd2020a3b57d3d8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:44.549477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-bd03c678-7114-40a7-9c78-cf464d77f932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '636885900e904df38bd2020a3b57d3d8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:44.567938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 845/1685] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:48:44 2022] DELETE /identity/v3/users/636885900e904df38bd2020a3b57d3d8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:44.572222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf47c4f-fa9c-48e5-ba87-e7ac9887b5ff None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:44.573325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf47c4f-fa9c-48e5-ba87-e7ac9887b5ff None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:44.573724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf47c4f-fa9c-48e5-ba87-e7ac9887b5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:44.625062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-edf47c4f-fa9c-48e5-ba87-e7ac9887b5ff None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:44.639722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-edf47c4f-fa9c-48e5-ba87-e7ac9887b5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:44.644157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-abe1487d-3ef0-4304-a6e1-1f1b9b88f56d None None] MFA Rules not processed for user `5eb7e8b4befd4d519b440c632b59e724`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:44.649272 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-abe1487d-3ef0-4304-a6e1-1f1b9b88f56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:44.863403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 846/1686] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:44.873949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 841/1687] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:44.888171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce6c24-6be8-4c81-869d-81d29287b579 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:44.888473 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce6c24-6be8-4c81-869d-81d29287b579 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:44.888818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce6c24-6be8-4c81-869d-81d29287b579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:44.891600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1487d-3ef0-4304-a6e1-1f1b9b88f56d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:44.893674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-abe1487d-3ef0-4304-a6e1-1f1b9b88f56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:44.937023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-33ce6c24-6be8-4c81-869d-81d29287b579 None None] MFA Rules not processed for user `5eb7e8b4befd4d519b440c632b59e724`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:44.938292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-33ce6c24-6be8-4c81-869d-81d29287b579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:44.968195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 847/1688] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:44.985685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1487d-3ef0-4304-a6e1-1f1b9b88f56d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:44.986931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce6c24-6be8-4c81-869d-81d29287b579 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:44.989605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-33ce6c24-6be8-4c81-869d-81d29287b579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:44.992810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3967809-23fe-4052-aa27-9f1fb4897dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:44.993924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3967809-23fe-4052-aa27-9f1fb4897dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:44.994492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3967809-23fe-4052-aa27-9f1fb4897dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:44.995132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3967809-23fe-4052-aa27-9f1fb4897dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:45.012574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3967809-23fe-4052-aa27-9f1fb4897dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:45.014700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3967809-23fe-4052-aa27-9f1fb4897dd2 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:45.036869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 842/1689] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:48:45.123538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce6c24-6be8-4c81-869d-81d29287b579 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:45.132129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b357192-47df-4f78-afea-9ddc2182841b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:45.132854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357192-47df-4f78-afea-9ddc2182841b service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:45.133141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357192-47df-4f78-afea-9ddc2182841b service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:45.133419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b357192-47df-4f78-afea-9ddc2182841b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:45.143500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b357192-47df-4f78-afea-9ddc2182841b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:45.145325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b357192-47df-4f78-afea-9ddc2182841b service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:45.168967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 848/1690] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:48:45.505967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3967809-23fe-4052-aa27-9f1fb4897dd2 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:45.509420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b3967809-23fe-4052-aa27-9f1fb4897dd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:45.535712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3967809-23fe-4052-aa27-9f1fb4897dd2 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:45.565122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:45.569064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:45.570724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:45.572007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] PATH_INFO: `/v3/projects/bba3035f3a99457eb9502a492612f37e` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:45.578073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba3035f3a99457eb9502a492612f37e)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:45.596360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:45.742007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba3035f3a99457eb9502a492612f37e', 'request_id': 'req-775813c3-463d-406a-85f2-8a44db127d8c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:45.762187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba3035f3a99457eb9502a492612f37e', 'request_id': 'req-775813c3-463d-406a-85f2-8a44db127d8c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:45.785251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba3035f3a99457eb9502a492612f37e', 'request_id': 'req-775813c3-463d-406a-85f2-8a44db127d8c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:45.794697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:45.813441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] The token cache is being invalidate because project bba3035f3a99457eb9502a492612f37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:48:45.813788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:45.815501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-775813c3-463d-406a-85f2-8a44db127d8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:45.821079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 843/1691] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:48:45 2022] DELETE /identity/v3/projects/bba3035f3a99457eb9502a492612f37e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:45.821851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d8a5e-f994-41de-88c5-4131e1ec5f46 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:45.822905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d8a5e-f994-41de-88c5-4131e1ec5f46 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:45.823410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d8a5e-f994-41de-88c5-4131e1ec5f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:45.957293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-aa8d8a5e-f994-41de-88c5-4131e1ec5f46 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:45.959481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-aa8d8a5e-f994-41de-88c5-4131e1ec5f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:46.056408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:46.223564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 849/1692] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:48:46.225528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b0b17-8d81-4d15-aeeb-536f512857e5 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:46.225992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b0b17-8d81-4d15-aeeb-536f512857e5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:46.226078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b0b17-8d81-4d15-aeeb-536f512857e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:46.267094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-533b0b17-8d81-4d15-aeeb-536f512857e5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:46.271998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-533b0b17-8d81-4d15-aeeb-536f512857e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:46.302585 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 844/1693] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:46.306810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d8a5e-f994-41de-88c5-4131e1ec5f46 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:46.326613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d8a5e-f994-41de-88c5-4131e1ec5f46 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:46.341831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c599696-f1ec-4aa1-be5a-2c84315ab3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:46.344236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c599696-f1ec-4aa1-be5a-2c84315ab3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:46.344916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c599696-f1ec-4aa1-be5a-2c84315ab3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:46.346130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c599696-f1ec-4aa1-be5a-2c84315ab3d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:46.348563 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c599696-f1ec-4aa1-be5a-2c84315ab3d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:46.349282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c599696-f1ec-4aa1-be5a-2c84315ab3d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:46.352875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c599696-f1ec-4aa1-be5a-2c84315ab3d9 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:46.366929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 850/1694] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:46.372055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b0b17-8d81-4d15-aeeb-536f512857e5 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:46.407816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b0b17-8d81-4d15-aeeb-536f512857e5 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:46.420104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be5504-705a-46c1-910c-08db210ca359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:46.422278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be5504-705a-46c1-910c-08db210ca359 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:46.422887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be5504-705a-46c1-910c-08db210ca359 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:46.424384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be5504-705a-46c1-910c-08db210ca359 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:46.426185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be5504-705a-46c1-910c-08db210ca359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:46.431008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be5504-705a-46c1-910c-08db210ca359 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:46.511202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 845/1695] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:46 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:48:46.516091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c599696-f1ec-4aa1-be5a-2c84315ab3d9 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:46.529490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c599696-f1ec-4aa1-be5a-2c84315ab3d9 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:46.551426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88b605-24ca-4a44-8e24-e2804421e56c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:46.552106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88b605-24ca-4a44-8e24-e2804421e56c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:46.552106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88b605-24ca-4a44-8e24-e2804421e56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:46.642596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-4f88b605-24ca-4a44-8e24-e2804421e56c None None] MFA Rules not processed for user `2e78cf97bac4484f93c7359b86a26628`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:46.653597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4f88b605-24ca-4a44-8e24-e2804421e56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:46.752236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b4d7c-e69c-450d-ad11-a5b14519d34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:46.753128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b4d7c-e69c-450d-ad11-a5b14519d34f admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:46.753431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b4d7c-e69c-450d-ad11-a5b14519d34f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:46.753689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b4d7c-e69c-450d-ad11-a5b14519d34f admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:46.754380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b4d7c-e69c-450d-ad11-a5b14519d34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:46.756762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b4d7c-e69c-450d-ad11-a5b14519d34f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:46.841764 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 851/1696] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:46 2022] POST /identity/v3/users => generated 291 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:46.841764 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b4d7c-e69c-450d-ad11-a5b14519d34f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:46.864283 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2f8b4d7c-e69c-450d-ad11-a5b14519d34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:46.876098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 846/1697] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:48:46.884035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b4d7c-e69c-450d-ad11-a5b14519d34f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:46.887349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88b605-24ca-4a44-8e24-e2804421e56c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:46.907111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:46.913284 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:46.913564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:46.913840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] PATH_INFO: `/v3/projects/b9f2b9eddebb4db68270964f0ba1b826` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:46.922418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f2b9eddebb4db68270964f0ba1b826)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:46.927283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88b605-24ca-4a44-8e24-e2804421e56c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:46.929905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:46.953092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7720bc-a861-4f6a-94b6-65c25666d99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:46.954091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7720bc-a861-4f6a-94b6-65c25666d99d admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:46.954566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7720bc-a861-4f6a-94b6-65c25666d99d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:46.955273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7720bc-a861-4f6a-94b6-65c25666d99d admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:46.961343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7720bc-a861-4f6a-94b6-65c25666d99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:46.963663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7720bc-a861-4f6a-94b6-65c25666d99d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:46.985931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 847/1698] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:46.987143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7720bc-a861-4f6a-94b6-65c25666d99d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:46.989365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-be7720bc-a861-4f6a-94b6-65c25666d99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:47.079539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f2b9eddebb4db68270964f0ba1b826', 'request_id': 'req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.112750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f2b9eddebb4db68270964f0ba1b826', 'request_id': 'req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.126754 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f2b9eddebb4db68270964f0ba1b826', 'request_id': 'req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.166546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] The token cache is being invalidate because project b9f2b9eddebb4db68270964f0ba1b826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:48:47.166853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.169880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.170891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7720bc-a861-4f6a-94b6-65c25666d99d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:47.175938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 852/1699] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:48:46 2022] DELETE /identity/v3/projects/b9f2b9eddebb4db68270964f0ba1b826 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:47.180490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:47.187822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:47.254335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:47.259870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:47.309839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d09857-a48b-4dd4-9f13-67ac0f91dde2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:47.310846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09857-a48b-4dd4-9f13-67ac0f91dde2 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:47.311437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09857-a48b-4dd4-9f13-67ac0f91dde2 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:47.312004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09857-a48b-4dd4-9f13-67ac0f91dde2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:47.314404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b4d09857-a48b-4dd4-9f13-67ac0f91dde2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:47.438506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faea1c6-cdb1-4262-abad-67b4d1c27d2e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:47.446511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837ae2d-a562-4b96-894d-228319377eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:47.448379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837ae2d-a562-4b96-894d-228319377eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:47.448781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837ae2d-a562-4b96-894d-228319377eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:47.449069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837ae2d-a562-4b96-894d-228319377eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:47.450774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837ae2d-a562-4b96-894d-228319377eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:47.452949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837ae2d-a562-4b96-894d-228319377eb1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:47.499231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 853/1700] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:47.503594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837ae2d-a562-4b96-894d-228319377eb1 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:47.505440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8837ae2d-a562-4b96-894d-228319377eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:47.515044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d09857-a48b-4dd4-9f13-67ac0f91dde2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:47.516906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d09857-a48b-4dd4-9f13-67ac0f91dde2 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:47.531508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837ae2d-a562-4b96-894d-228319377eb1 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:47.549735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 848/1701] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:47.551271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a3f6-1fb0-4557-99af-7c85df08ea57 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:47.551435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a3f6-1fb0-4557-99af-7c85df08ea57 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:47.551717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66a3f6-1fb0-4557-99af-7c85df08ea57 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:47.554165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 849/1702] 10.222.0.47 () {54 vars in 912 bytes} [Fri Jun 24 23:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:48:47.555754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66a3f6-1fb0-4557-99af-7c85df08ea57 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:47.556838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:47.557622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:47.557902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:47.558183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] PATH_INFO: `/v3/users/933df08a4d3b471aafe9f0826133e6ac` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:47.559832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=933df08a4d3b471aafe9f0826133e6ac)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:47.561845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:47.594181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66a3f6-1fb0-4557-99af-7c85df08ea57 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:47.607926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817cda7-ec24-469b-9109-eb78fba69d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:47.611512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817cda7-ec24-469b-9109-eb78fba69d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:47.611804 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817cda7-ec24-469b-9109-eb78fba69d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:47.612080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817cda7-ec24-469b-9109-eb78fba69d7d admin admin] PATH_INFO: `/v3/projects/05d1cda6e3904da5824a78fb8defbd8f/users/9f18cf838a8e43348afe3d68fa799d20/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:47.710228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817cda7-ec24-469b-9109-eb78fba69d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d1cda6e3904da5824a78fb8defbd8f, user_id=9f18cf838a8e43348afe3d68fa799d20, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:47.747382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933df08a4d3b471aafe9f0826133e6ac'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.760031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817cda7-ec24-469b-9109-eb78fba69d7d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:47.777964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933df08a4d3b471aafe9f0826133e6ac'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.783123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933df08a4d3b471aafe9f0826133e6ac'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.785473 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933df08a4d3b471aafe9f0826133e6ac'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.811024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 850/1703] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:47 2022] PUT /identity/v3/projects/05d1cda6e3904da5824a78fb8defbd8f/users/9f18cf838a8e43348afe3d68fa799d20/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:47.811222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3250b06d-5952-4a45-8936-a0683b30e6e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933df08a4d3b471aafe9f0826133e6ac'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:47.817830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d627ca3-aebe-4690-882f-2b13fedacc27 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:47.818417 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d627ca3-aebe-4690-882f-2b13fedacc27 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:47.818878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d627ca3-aebe-4690-882f-2b13fedacc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:47.864493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 854/1704] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:48:47 2022] DELETE /identity/v3/users/933df08a4d3b471aafe9f0826133e6ac => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:47.868118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab371889-e538-4874-b14a-bdc19b8d33c5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:47.868502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab371889-e538-4874-b14a-bdc19b8d33c5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:47.869419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab371889-e538-4874-b14a-bdc19b8d33c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:47.954562 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-3d627ca3-aebe-4690-882f-2b13fedacc27 None None] MFA Rules not processed for user `e38a311f2b614a3598f6b51b3a235178`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:47.956435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3d627ca3-aebe-4690-882f-2b13fedacc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:47.988687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ab371889-e538-4874-b14a-bdc19b8d33c5 None None] MFA Rules not processed for user `9f18cf838a8e43348afe3d68fa799d20`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:47.990060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ab371889-e538-4874-b14a-bdc19b8d33c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:48.138760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 851/1705] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:48:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:48.139160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c0348-2dde-40fd-9144-6b41990bf470 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:48.139454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c0348-2dde-40fd-9144-6b41990bf470 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:48.141144 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c0348-2dde-40fd-9144-6b41990bf470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:48.160307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 855/1706] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:48.164784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab371889-e538-4874-b14a-bdc19b8d33c5 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:48.169291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ab371889-e538-4874-b14a-bdc19b8d33c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:48.224457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-4d1c0348-2dde-40fd-9144-6b41990bf470 None None] MFA Rules not processed for user `927ffbcd7078473f97f42bdb54acf57e`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:48.226229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4d1c0348-2dde-40fd-9144-6b41990bf470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:48.231793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:48.347958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:48.379927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab371889-e538-4874-b14a-bdc19b8d33c5 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:48.385219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e897afe-296a-4629-b3bb-4c368c7d9119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:48.385897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e897afe-296a-4629-b3bb-4c368c7d9119 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:48.386174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e897afe-296a-4629-b3bb-4c368c7d9119 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:48.386444 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e897afe-296a-4629-b3bb-4c368c7d9119 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:48.387184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e897afe-296a-4629-b3bb-4c368c7d9119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:48.392771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e897afe-296a-4629-b3bb-4c368c7d9119 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:48.405413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 852/1707] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:48.406316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c0348-2dde-40fd-9144-6b41990bf470 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:48.409967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4d1c0348-2dde-40fd-9144-6b41990bf470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:48.462919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 856/1708] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:48 2022] POST /identity/v3/projects => generated 379 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:48.463811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e897afe-296a-4629-b3bb-4c368c7d9119 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:48.467845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c0348-2dde-40fd-9144-6b41990bf470 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:48.469981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3e897afe-296a-4629-b3bb-4c368c7d9119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:48.567636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ff4b9-1b0a-40b8-ac05-d808516434f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:48.568476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ff4b9-1b0a-40b8-ac05-d808516434f7 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:48.568811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ff4b9-1b0a-40b8-ac05-d808516434f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:48.569102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ff4b9-1b0a-40b8-ac05-d808516434f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:48.586544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ff4b9-1b0a-40b8-ac05-d808516434f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:48.588017 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ff4b9-1b0a-40b8-ac05-d808516434f7 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:48.608427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 853/1709] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:48.609396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ff4b9-1b0a-40b8-ac05-d808516434f7 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:48.711259 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e897afe-296a-4629-b3bb-4c368c7d9119 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:48.716574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ff4b9-1b0a-40b8-ac05-d808516434f7 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:48.723085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70ad12-ed4c-492c-9a0f-597d40248e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:48.724207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70ad12-ed4c-492c-9a0f-597d40248e64 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:48.724550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70ad12-ed4c-492c-9a0f-597d40248e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:48.724869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70ad12-ed4c-492c-9a0f-597d40248e64 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:48.731415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70ad12-ed4c-492c-9a0f-597d40248e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:48.731847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee33357-3780-4e13-bbab-906ef404eeb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:48.732799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee33357-3780-4e13-bbab-906ef404eeb5 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:48.734187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee33357-3780-4e13-bbab-906ef404eeb5 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:48.734592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee33357-3780-4e13-bbab-906ef404eeb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:48.735200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70ad12-ed4c-492c-9a0f-597d40248e64 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:48.770395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 854/1710] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:48 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:48.770395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70ad12-ed4c-492c-9a0f-597d40248e64 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:48.773571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4b70ad12-ed4c-492c-9a0f-597d40248e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:48.819378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70ad12-ed4c-492c-9a0f-597d40248e64 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:48.835362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee33357-3780-4e13-bbab-906ef404eeb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:48.837335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee33357-3780-4e13-bbab-906ef404eeb5 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:48.844138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:48.845072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:48.845354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:48.845631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] PATH_INFO: `/v3/projects/9985ebd0b5464d34bb07758f80bd180a` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:48.847656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9985ebd0b5464d34bb07758f80bd180a)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:48.849818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:48.865568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 857/1711] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:48.869088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee33357-3780-4e13-bbab-906ef404eeb5 service nova] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:48.890406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee33357-3780-4e13-bbab-906ef404eeb5 service nova] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:48.905167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a9e5f-c38d-4360-bd69-2d2f9764755c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:48.906029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9e5f-c38d-4360-bd69-2d2f9764755c admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:48.906569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9e5f-c38d-4360-bd69-2d2f9764755c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:48.907168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9e5f-c38d-4360-bd69-2d2f9764755c admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:48.908388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a9e5f-c38d-4360-bd69-2d2f9764755c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:48.910407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a9e5f-c38d-4360-bd69-2d2f9764755c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:48.933162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 858/1712] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:48.937470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a9e5f-c38d-4360-bd69-2d2f9764755c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:48.957958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a9e5f-c38d-4360-bd69-2d2f9764755c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:48.986722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9985ebd0b5464d34bb07758f80bd180a', 'request_id': 'req-0ea4a180-b7c7-4fff-8643-38e6187c1a78'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:49.015587 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9985ebd0b5464d34bb07758f80bd180a', 'request_id': 'req-0ea4a180-b7c7-4fff-8643-38e6187c1a78'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:49.038763 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9985ebd0b5464d34bb07758f80bd180a', 'request_id': 'req-0ea4a180-b7c7-4fff-8643-38e6187c1a78'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:49.040135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:49.063589 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:49.077306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] The token cache is being invalidate because project 9985ebd0b5464d34bb07758f80bd180a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:48:49.078197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:49.080099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0ea4a180-b7c7-4fff-8643-38e6187c1a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:48:49.086369 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 855/1713] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:48:48 2022] DELETE /identity/v3/projects/9985ebd0b5464d34bb07758f80bd180a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:49.094854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e77b78-6988-4e94-bb40-79dfdd517fee None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:49.095636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e77b78-6988-4e94-bb40-79dfdd517fee None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:49.099328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e77b78-6988-4e94-bb40-79dfdd517fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:49.126132 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451a8f3-230d-410f-92b7-927a135e121a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:49.127357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451a8f3-230d-410f-92b7-927a135e121a admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:49.127789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451a8f3-230d-410f-92b7-927a135e121a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:49.128315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451a8f3-230d-410f-92b7-927a135e121a admin admin] PATH_INFO: `/v3/projects/c69cb736720b4f6eb0f1f4ef4bde3131/users/90117311e5b34ac68c112923e7e56f36/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:49.147980 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451a8f3-230d-410f-92b7-927a135e121a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69cb736720b4f6eb0f1f4ef4bde3131, user_id=90117311e5b34ac68c112923e7e56f36, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:49.154385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451a8f3-230d-410f-92b7-927a135e121a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:49.187867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 859/1714] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:48 2022] PUT /identity/v3/projects/c69cb736720b4f6eb0f1f4ef4bde3131/users/90117311e5b34ac68c112923e7e56f36/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:49.192363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451a8f3-230d-410f-92b7-927a135e121a admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:49.193582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-04e77b78-6988-4e94-bb40-79dfdd517fee None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:49.196177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-04e77b78-6988-4e94-bb40-79dfdd517fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:49.196510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2451a8f3-230d-410f-92b7-927a135e121a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:49.347461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451a8f3-230d-410f-92b7-927a135e121a admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:49.352693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51501c09-611d-4bd5-8185-df14b01b6103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:49.354579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51501c09-611d-4bd5-8185-df14b01b6103 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:49.354914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51501c09-611d-4bd5-8185-df14b01b6103 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:49.355254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51501c09-611d-4bd5-8185-df14b01b6103 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:49.356471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51501c09-611d-4bd5-8185-df14b01b6103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:49.358678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51501c09-611d-4bd5-8185-df14b01b6103 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:49.380405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 860/1715] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:49.384452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51501c09-611d-4bd5-8185-df14b01b6103 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:49.392792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 856/1716] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:49.395977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6ceaf-05b5-4293-826a-e8454f4ca7df None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:49.396321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6ceaf-05b5-4293-826a-e8454f4ca7df None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:49.396631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6ceaf-05b5-4293-826a-e8454f4ca7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:49.405496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51501c09-611d-4bd5-8185-df14b01b6103 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:49.420889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679927e-0608-403f-93c4-d8e0a2078742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:49.421801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679927e-0608-403f-93c4-d8e0a2078742 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:49.422087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679927e-0608-403f-93c4-d8e0a2078742 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:49.422365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679927e-0608-403f-93c4-d8e0a2078742 admin admin] PATH_INFO: `/v3/projects/c69cb736720b4f6eb0f1f4ef4bde3131/users/90117311e5b34ac68c112923e7e56f36/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:49.430919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679927e-0608-403f-93c4-d8e0a2078742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69cb736720b4f6eb0f1f4ef4bde3131, user_id=90117311e5b34ac68c112923e7e56f36, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:49.434419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679927e-0608-403f-93c4-d8e0a2078742 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:49.462493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-84a6ceaf-05b5-4293-826a-e8454f4ca7df None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:49.466631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-84a6ceaf-05b5-4293-826a-e8454f4ca7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:49.484931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 861/1717] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:49 2022] PUT /identity/v3/projects/c69cb736720b4f6eb0f1f4ef4bde3131/users/90117311e5b34ac68c112923e7e56f36/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:49.497422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6c589-b34f-43ad-9c8a-126cd8e892ba None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:49.497943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6c589-b34f-43ad-9c8a-126cd8e892ba None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:49.498256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6c589-b34f-43ad-9c8a-126cd8e892ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:49.588660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6fa6c589-b34f-43ad-9c8a-126cd8e892ba None None] MFA Rules not processed for user `90117311e5b34ac68c112923e7e56f36`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:49.590376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6fa6c589-b34f-43ad-9c8a-126cd8e892ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:49.744346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 857/1718] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:49.751537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6ceaf-05b5-4293-826a-e8454f4ca7df None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:49.776894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6ceaf-05b5-4293-826a-e8454f4ca7df None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:49.788888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d09b-527d-4c54-bb33-43dbd01b5756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:49.789778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d09b-527d-4c54-bb33-43dbd01b5756 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:49.790070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d09b-527d-4c54-bb33-43dbd01b5756 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:49.791676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90d09b-527d-4c54-bb33-43dbd01b5756 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:49.792503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90d09b-527d-4c54-bb33-43dbd01b5756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:48:49.794056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90d09b-527d-4c54-bb33-43dbd01b5756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:49.797865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90d09b-527d-4c54-bb33-43dbd01b5756 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:49.834026 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 858/1719] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:48:49.836678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d09b-527d-4c54-bb33-43dbd01b5756 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:49.870894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90d09b-527d-4c54-bb33-43dbd01b5756 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:49.879813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 862/1720] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:49.888353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6c589-b34f-43ad-9c8a-126cd8e892ba None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:49.889820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6fa6c589-b34f-43ad-9c8a-126cd8e892ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:49.891022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437a923-aba9-4726-8402-9786d5bb2d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:49.895070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437a923-aba9-4726-8402-9786d5bb2d63 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:49.895610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437a923-aba9-4726-8402-9786d5bb2d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:49.902314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437a923-aba9-4726-8402-9786d5bb2d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:49.903616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437a923-aba9-4726-8402-9786d5bb2d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:49.909985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437a923-aba9-4726-8402-9786d5bb2d63 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:49.934654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:49.986016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 859/1721] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:49 2022] POST /identity/v3/projects => generated 393 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:49.987321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437a923-aba9-4726-8402-9786d5bb2d63 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:50.028131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6c589-b34f-43ad-9c8a-126cd8e892ba None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:50.133295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437a923-aba9-4726-8402-9786d5bb2d63 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:50.140292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cffa3-f601-4ce2-97aa-0a3a79454d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:50.142608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cffa3-f601-4ce2-97aa-0a3a79454d31 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:50.143010 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cffa3-f601-4ce2-97aa-0a3a79454d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:50.145216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cffa3-f601-4ce2-97aa-0a3a79454d31 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:50.146152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cffa3-f601-4ce2-97aa-0a3a79454d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:50.160534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cffa3-f601-4ce2-97aa-0a3a79454d31 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:50.165414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d8822-5ec7-47e6-9da1-86ffece64926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:50.166334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d8822-5ec7-47e6-9da1-86ffece64926 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:50.166757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d8822-5ec7-47e6-9da1-86ffece64926 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:50.167190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d8822-5ec7-47e6-9da1-86ffece64926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:50.174400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:50.202315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 860/1722] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:49 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:50.207630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cffa3-f601-4ce2-97aa-0a3a79454d31 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:50.232472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cffa3-f601-4ce2-97aa-0a3a79454d31 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:50.241582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf20444-c4a1-407c-8078-0732bf3c669e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:50.242398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf20444-c4a1-407c-8078-0732bf3c669e admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:50.242786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf20444-c4a1-407c-8078-0732bf3c669e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:50.243265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf20444-c4a1-407c-8078-0732bf3c669e admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:50.244703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf20444-c4a1-407c-8078-0732bf3c669e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:50.247311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf20444-c4a1-407c-8078-0732bf3c669e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:50.265214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 861/1723] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:50.266323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf20444-c4a1-407c-8078-0732bf3c669e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:50.291157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf20444-c4a1-407c-8078-0732bf3c669e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:50.316817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa6cd7-ed5e-4b5a-8efc-f28a14f88759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:50.316817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa6cd7-ed5e-4b5a-8efc-f28a14f88759 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:50.316817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa6cd7-ed5e-4b5a-8efc-f28a14f88759 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:50.316817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa6cd7-ed5e-4b5a-8efc-f28a14f88759 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:50.319026 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa6cd7-ed5e-4b5a-8efc-f28a14f88759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:50.326326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa6cd7-ed5e-4b5a-8efc-f28a14f88759 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:50.368524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 862/1724] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:50.370920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d8822-5ec7-47e6-9da1-86ffece64926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:50.371886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa6cd7-ed5e-4b5a-8efc-f28a14f88759 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:50.375462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d8822-5ec7-47e6-9da1-86ffece64926 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:50.400773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa6cd7-ed5e-4b5a-8efc-f28a14f88759 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:50.409279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 863/1725] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:50.411997 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df81ac-eab6-4f72-b5e9-0d9b2bfaff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:50.413661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df81ac-eab6-4f72-b5e9-0d9b2bfaff1d admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:50.414187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df81ac-eab6-4f72-b5e9-0d9b2bfaff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:50.414580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df81ac-eab6-4f72-b5e9-0d9b2bfaff1d admin admin] PATH_INFO: `/v3/projects/a4b80b9ac84b47d98df3eab189fb64e8/users/a6a9ea67006d4f8c9c8218ba36767a06/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:50.436439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df81ac-eab6-4f72-b5e9-0d9b2bfaff1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b80b9ac84b47d98df3eab189fb64e8, user_id=a6a9ea67006d4f8c9c8218ba36767a06, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:50.440714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df81ac-eab6-4f72-b5e9-0d9b2bfaff1d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:50.490427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 863/1726] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:50 2022] PUT /identity/v3/projects/a4b80b9ac84b47d98df3eab189fb64e8/users/a6a9ea67006d4f8c9c8218ba36767a06/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:50.503271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e966f80-fb09-48bb-a3cc-3c28001db240 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:50.503613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e966f80-fb09-48bb-a3cc-3c28001db240 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:50.503920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e966f80-fb09-48bb-a3cc-3c28001db240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:50.565945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df81ac-eab6-4f72-b5e9-0d9b2bfaff1d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:50.580106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0e966f80-fb09-48bb-a3cc-3c28001db240 None None] MFA Rules not processed for user `a6a9ea67006d4f8c9c8218ba36767a06`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:50.581427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0e966f80-fb09-48bb-a3cc-3c28001db240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:50.790452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 864/1727] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:50.795533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e966f80-fb09-48bb-a3cc-3c28001db240 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:50.871776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df81ac-eab6-4f72-b5e9-0d9b2bfaff1d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:50.874889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:50.894688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9379483-ae16-4331-9ed1-db553b9b6247 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:50.895525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379483-ae16-4331-9ed1-db553b9b6247 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:50.898394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379483-ae16-4331-9ed1-db553b9b6247 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:50.900934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9379483-ae16-4331-9ed1-db553b9b6247 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:50.909660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a9379483-ae16-4331-9ed1-db553b9b6247 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:51.076154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e966f80-fb09-48bb-a3cc-3c28001db240 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:51.091346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c509b-45dc-4856-b5f1-d327c8275ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:51.092301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c509b-45dc-4856-b5f1-d327c8275ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:51.095215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c509b-45dc-4856-b5f1-d327c8275ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:51.095682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c509b-45dc-4856-b5f1-d327c8275ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:51.096573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c509b-45dc-4856-b5f1-d327c8275ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:51.103607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c509b-45dc-4856-b5f1-d327c8275ce6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:51.113773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9379483-ae16-4331-9ed1-db553b9b6247 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:51.116038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9379483-ae16-4331-9ed1-db553b9b6247 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:51.140706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 864/1728] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:51.221029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 865/1729] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:48:50 2022] POST /identity/v3/projects => generated 393 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:51.234197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9379483-ae16-4331-9ed1-db553b9b6247 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:51.268639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9379483-ae16-4331-9ed1-db553b9b6247 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:51.404565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599095a-51d7-4143-b46f-75583e31b786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:51.405459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599095a-51d7-4143-b46f-75583e31b786 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:51.405876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599095a-51d7-4143-b46f-75583e31b786 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:51.406160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599095a-51d7-4143-b46f-75583e31b786 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:51.407070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599095a-51d7-4143-b46f-75583e31b786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:51.409277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599095a-51d7-4143-b46f-75583e31b786 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:51.424576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:51.444375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79cad0-6fc6-4913-a25a-435ec2732177 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:51.444937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79cad0-6fc6-4913-a25a-435ec2732177 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:51.445142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb79cad0-6fc6-4913-a25a-435ec2732177 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:51.446902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 866/1730] 10.222.0.47 () {54 vars in 960 bytes} [Fri Jun 24 23:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 23:48:51.459056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ac157-8171-4f5c-8c86-0808be947882 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:51.465676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ac157-8171-4f5c-8c86-0808be947882 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:51.466060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ac157-8171-4f5c-8c86-0808be947882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:51.486251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 865/1731] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:48:51 2022] POST /identity/v3/users => generated 303 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:48:51.488696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599095a-51d7-4143-b46f-75583e31b786 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:51.533001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599095a-51d7-4143-b46f-75583e31b786 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:51.534494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-980ac157-8171-4f5c-8c86-0808be947882 None None] MFA Rules not processed for user `a243d444b7ae487cb0331523d4404d19`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:51.536821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-980ac157-8171-4f5c-8c86-0808be947882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:51.555324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8943c0-ec62-45a1-912a-1a46eee03223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:51.556110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8943c0-ec62-45a1-912a-1a46eee03223 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:51.556427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8943c0-ec62-45a1-912a-1a46eee03223 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:51.556744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8943c0-ec62-45a1-912a-1a46eee03223 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:51.557482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8943c0-ec62-45a1-912a-1a46eee03223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:51.560139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8943c0-ec62-45a1-912a-1a46eee03223 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:51.644325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 866/1732] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:51.649336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8943c0-ec62-45a1-912a-1a46eee03223 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:51.681086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8943c0-ec62-45a1-912a-1a46eee03223 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:51.697711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644017a4-6469-4de8-9249-a03218ae5a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:51.699137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644017a4-6469-4de8-9249-a03218ae5a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:51.700139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644017a4-6469-4de8-9249-a03218ae5a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:51.702755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644017a4-6469-4de8-9249-a03218ae5a6b admin admin] PATH_INFO: `/v3/projects/ef17c9b83716491388c4f9cf91106158/users/64cdd10f43e84b76b40f11f9e0026935/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:51.786354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644017a4-6469-4de8-9249-a03218ae5a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef17c9b83716491388c4f9cf91106158, user_id=64cdd10f43e84b76b40f11f9e0026935, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:51.799180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644017a4-6469-4de8-9249-a03218ae5a6b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:51.822409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 867/1733] 10.222.0.47 () {58 vars in 1057 bytes} [Fri Jun 24 23:48:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:51.832650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c2d9d-9a09-4eeb-a04e-5b53b056b282 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:51.834359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c2d9d-9a09-4eeb-a04e-5b53b056b282 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:51.836463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c2d9d-9a09-4eeb-a04e-5b53b056b282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:51.850623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 867/1734] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:51 2022] PUT /identity/v3/projects/ef17c9b83716491388c4f9cf91106158/users/64cdd10f43e84b76b40f11f9e0026935/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:51.852655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644017a4-6469-4de8-9249-a03218ae5a6b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:51.901798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-713c2d9d-9a09-4eeb-a04e-5b53b056b282 None None] MFA Rules not processed for user `5eb7e8b4befd4d519b440c632b59e724`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:51.908664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-713c2d9d-9a09-4eeb-a04e-5b53b056b282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:52.090318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644017a4-6469-4de8-9249-a03218ae5a6b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:52.103379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb4856-9dda-4f2c-91ce-5fd6aea785ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:52.104061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb4856-9dda-4f2c-91ce-5fd6aea785ee service placement] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:52.104515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb4856-9dda-4f2c-91ce-5fd6aea785ee service placement] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:52.104989 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb4856-9dda-4f2c-91ce-5fd6aea785ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:52.108349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-94cb4856-9dda-4f2c-91ce-5fd6aea785ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:52.116745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 868/1735] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:52.117654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c2d9d-9a09-4eeb-a04e-5b53b056b282 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:52.132814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb4856-9dda-4f2c-91ce-5fd6aea785ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:52.136097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb4856-9dda-4f2c-91ce-5fd6aea785ee service placement] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:52.183117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 868/1736] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:52.382978 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:52.464137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c2d9d-9a09-4eeb-a04e-5b53b056b282 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:52.482001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4c725-a98f-4eeb-8820-4f880f8e053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:52.487535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4c725-a98f-4eeb-8820-4f880f8e053c admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:52.488232 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4c725-a98f-4eeb-8820-4f880f8e053c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:52.491786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4c725-a98f-4eeb-8820-4f880f8e053c admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:52.492775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4c725-a98f-4eeb-8820-4f880f8e053c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:52.501587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4c725-a98f-4eeb-8820-4f880f8e053c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:52.549757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 869/1737] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:48:52.554290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb4856-9dda-4f2c-91ce-5fd6aea785ee service placement] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:52.581147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb4856-9dda-4f2c-91ce-5fd6aea785ee service placement] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:52.609465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad9a88-dd92-47f5-8f3a-dd9b0a0bae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:52.611660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad9a88-dd92-47f5-8f3a-dd9b0a0bae06 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:52.612687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad9a88-dd92-47f5-8f3a-dd9b0a0bae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:52.613862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad9a88-dd92-47f5-8f3a-dd9b0a0bae06 admin admin] PATH_INFO: `/v3/projects/ef17c9b83716491388c4f9cf91106158/users/64cdd10f43e84b76b40f11f9e0026935/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:52.625343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad9a88-dd92-47f5-8f3a-dd9b0a0bae06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef17c9b83716491388c4f9cf91106158, user_id=64cdd10f43e84b76b40f11f9e0026935, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:52.633973 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad9a88-dd92-47f5-8f3a-dd9b0a0bae06 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:52.677717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 869/1738] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:48:52 2022] PUT /identity/v3/projects/ef17c9b83716491388c4f9cf91106158/users/64cdd10f43e84b76b40f11f9e0026935/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:48:52.689178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89a773-7c53-4ddf-b744-00c2997586e3 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:52.689791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89a773-7c53-4ddf-b744-00c2997586e3 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:52.690555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89a773-7c53-4ddf-b744-00c2997586e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:52.793994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ae89a773-7c53-4ddf-b744-00c2997586e3 None None] MFA Rules not processed for user `64cdd10f43e84b76b40f11f9e0026935`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:52.795331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ae89a773-7c53-4ddf-b744-00c2997586e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:53.090318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 870/1739] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:53.103368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad9a88-dd92-47f5-8f3a-dd9b0a0bae06 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:53.105705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-44ad9a88-dd92-47f5-8f3a-dd9b0a0bae06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:53.328345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad9a88-dd92-47f5-8f3a-dd9b0a0bae06 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:53.332522 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb55ef1-99cf-48f8-8c62-a32481057db9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:53.333285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55ef1-99cf-48f8-8c62-a32481057db9 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:53.333573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55ef1-99cf-48f8-8c62-a32481057db9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:53.333846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55ef1-99cf-48f8-8c62-a32481057db9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:53.363572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55ef1-99cf-48f8-8c62-a32481057db9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:53.375436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb55ef1-99cf-48f8-8c62-a32481057db9 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:53.407070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 870/1740] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:53.573853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89a773-7c53-4ddf-b744-00c2997586e3 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:53.587232 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89a773-7c53-4ddf-b744-00c2997586e3 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:53.598743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f98257-9122-4b19-ad4d-612fe68204ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:53.599554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f98257-9122-4b19-ad4d-612fe68204ac service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:53.602314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f98257-9122-4b19-ad4d-612fe68204ac service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:53.602736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f98257-9122-4b19-ad4d-612fe68204ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:53.832512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f98257-9122-4b19-ad4d-612fe68204ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:53.837417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f98257-9122-4b19-ad4d-612fe68204ac service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:53.877347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 871/1741] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:54.258395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c731625-17e6-4a8a-81eb-bba6f39150d0 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:54.260827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c731625-17e6-4a8a-81eb-bba6f39150d0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:54.260827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c731625-17e6-4a8a-81eb-bba6f39150d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:54.320147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-3c731625-17e6-4a8a-81eb-bba6f39150d0 None None] MFA Rules not processed for user `151db87b1e7248d882108519fab1d938`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:54.330391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3c731625-17e6-4a8a-81eb-bba6f39150d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:54.378586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b312922-ef60-486a-846d-1bb17b1ba40a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:54.378886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b312922-ef60-486a-846d-1bb17b1ba40a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:54.379169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b312922-ef60-486a-846d-1bb17b1ba40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:54.397092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:54.427701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-4b312922-ef60-486a-846d-1bb17b1ba40a None None] MFA Rules not processed for user `2e78cf97bac4484f93c7359b86a26628`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:54.433218 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4b312922-ef60-486a-846d-1bb17b1ba40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:54.531373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 871/1742] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:54.543400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57619523-b124-4765-a172-18e9e13ead3b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:54.543869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57619523-b124-4765-a172-18e9e13ead3b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:54.544335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57619523-b124-4765-a172-18e9e13ead3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:54.613123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-57619523-b124-4765-a172-18e9e13ead3b None None] MFA Rules not processed for user `151db87b1e7248d882108519fab1d938`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:54.620084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-57619523-b124-4765-a172-18e9e13ead3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:54.659379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 872/1743] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:54.672082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57619523-b124-4765-a172-18e9e13ead3b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:54.674488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-57619523-b124-4765-a172-18e9e13ead3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:54.684445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 872/1744] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:54.901861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57619523-b124-4765-a172-18e9e13ead3b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:54.941710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680619cc-f00a-481a-8f7c-6cdbb744dfaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:54.942229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680619cc-f00a-481a-8f7c-6cdbb744dfaa service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:54.942642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680619cc-f00a-481a-8f7c-6cdbb744dfaa service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:54.943067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680619cc-f00a-481a-8f7c-6cdbb744dfaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:54.963983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680619cc-f00a-481a-8f7c-6cdbb744dfaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:54.966338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680619cc-f00a-481a-8f7c-6cdbb744dfaa service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:54.977241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 873/1745] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:55.155261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e4be1-42c0-49c0-9714-71ddffcafaf7 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:55.155708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e4be1-42c0-49c0-9714-71ddffcafaf7 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:55.156117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e4be1-42c0-49c0-9714-71ddffcafaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:55.174813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:55.271035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9b9e4be1-42c0-49c0-9714-71ddffcafaf7 None None] MFA Rules not processed for user `04e35f840ef045f0b44fab3e57a1a3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:55.281897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9b9e4be1-42c0-49c0-9714-71ddffcafaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:55.560752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 873/1746] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:56.073268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d6de8-4c6f-44fe-81ba-68703bd4722a None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:56.073795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d6de8-4c6f-44fe-81ba-68703bd4722a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:56.074129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d6de8-4c6f-44fe-81ba-68703bd4722a None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:56.075941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 874/1747] 10.222.0.47 () {54 vars in 912 bytes} [Fri Jun 24 23:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:48:56.081851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ff29-7807-4689-93e1-a4bd15ee7c7c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:56.083411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ff29-7807-4689-93e1-a4bd15ee7c7c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:56.083707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ff29-7807-4689-93e1-a4bd15ee7c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:56.107710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-4e45ff29-7807-4689-93e1-a4bd15ee7c7c None None] MFA Rules not processed for user `e38a311f2b614a3598f6b51b3a235178`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:56.110367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4e45ff29-7807-4689-93e1-a4bd15ee7c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:56.253030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 874/1748] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:48:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:56.270639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d6de8-4c6f-44fe-81ba-68703bd4722a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:56.272697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-cb5d6de8-4c6f-44fe-81ba-68703bd4722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:56.300687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d6de8-4c6f-44fe-81ba-68703bd4722a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:56.306549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf26343-9395-43d2-abc2-1e9bb8ee7150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:56.307227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf26343-9395-43d2-abc2-1e9bb8ee7150 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:56.307537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf26343-9395-43d2-abc2-1e9bb8ee7150 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:56.307818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf26343-9395-43d2-abc2-1e9bb8ee7150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:56.330746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf26343-9395-43d2-abc2-1e9bb8ee7150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:56.334467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf26343-9395-43d2-abc2-1e9bb8ee7150 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:56.359502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 875/1749] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:56.573212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48be51-3895-402f-80fe-62480a23f17c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:56.573541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48be51-3895-402f-80fe-62480a23f17c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:56.573783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48be51-3895-402f-80fe-62480a23f17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:56.631111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1d48be51-3895-402f-80fe-62480a23f17c None None] MFA Rules not processed for user `927ffbcd7078473f97f42bdb54acf57e`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:56.639215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1d48be51-3895-402f-80fe-62480a23f17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:56.836699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 875/1750] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:48:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:56.900545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf26343-9395-43d2-abc2-1e9bb8ee7150 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:56.901895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-faf26343-9395-43d2-abc2-1e9bb8ee7150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:56.943667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf26343-9395-43d2-abc2-1e9bb8ee7150 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:56.958289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c8814-2741-4395-b41e-2833b748b9d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:56.960782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c8814-2741-4395-b41e-2833b748b9d4 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:56.961091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c8814-2741-4395-b41e-2833b748b9d4 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:56.961375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c8814-2741-4395-b41e-2833b748b9d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:56.986468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c8814-2741-4395-b41e-2833b748b9d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:56.988245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c8814-2741-4395-b41e-2833b748b9d4 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:57.045649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 876/1751] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:48:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:48:58.774493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce690da5-ef2c-44fe-91b3-fbd8958f1ec0 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:58.775251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce690da5-ef2c-44fe-91b3-fbd8958f1ec0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:58.775251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce690da5-ef2c-44fe-91b3-fbd8958f1ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:58.826071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ce690da5-ef2c-44fe-91b3-fbd8958f1ec0 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:48:58.827775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ce690da5-ef2c-44fe-91b3-fbd8958f1ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:58.899876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:48:58.964465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 876/1752] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:48:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:48:59.250408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c8814-2741-4395-b41e-2833b748b9d4 service nova] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:48:59.251779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1d1c8814-2741-4395-b41e-2833b748b9d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:48:59.497010 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c8814-2741-4395-b41e-2833b748b9d4 service nova] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:48:59.524315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39c1fd-13cd-4140-a0bf-43ed4006d0bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:48:59.525044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c1fd-13cd-4140-a0bf-43ed4006d0bf service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:48:59.525671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c1fd-13cd-4140-a0bf-43ed4006d0bf service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:48:59.526092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39c1fd-13cd-4140-a0bf-43ed4006d0bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:48:59.549366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39c1fd-13cd-4140-a0bf-43ed4006d0bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:48:59.552679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39c1fd-13cd-4140-a0bf-43ed4006d0bf service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:48:59.597484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 877/1753] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:48:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:48:59.875071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce690da5-ef2c-44fe-91b3-fbd8958f1ec0 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:00.077805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce690da5-ef2c-44fe-91b3-fbd8958f1ec0 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:00.087833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:00.089088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:00.090152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:00.090742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] PATH_INFO: `/v3/users/a6a9ea67006d4f8c9c8218ba36767a06` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:00.092327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a9ea67006d4f8c9c8218ba36767a06)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:00.094543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:00.325696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a9ea67006d4f8c9c8218ba36767a06'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:00.420248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a9ea67006d4f8c9c8218ba36767a06'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:00.424001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a9ea67006d4f8c9c8218ba36767a06'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:00.425362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a9ea67006d4f8c9c8218ba36767a06'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:00.452465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a9ea67006d4f8c9c8218ba36767a06'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:00.525082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 877/1754] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:48:59 2022] DELETE /identity/v3/users/a6a9ea67006d4f8c9c8218ba36767a06 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:00.529283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db716135-c951-4f95-99ce-3da7ead325f8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:00.530345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db716135-c951-4f95-99ce-3da7ead325f8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:00.530647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db716135-c951-4f95-99ce-3da7ead325f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:00.602083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-db716135-c951-4f95-99ce-3da7ead325f8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:00.603501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-db716135-c951-4f95-99ce-3da7ead325f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:00.625043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:00.925507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 878/1755] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:01.299341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:01.301559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:01.560107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe37d9-27f7-4c7d-835a-192e89b27e78 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:01.578516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dda944-19ad-4277-94d8-3512ed8c4fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:01.579645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dda944-19ad-4277-94d8-3512ed8c4fbf service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:01.580075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dda944-19ad-4277-94d8-3512ed8c4fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:01.580511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dda944-19ad-4277-94d8-3512ed8c4fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:01.614869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dda944-19ad-4277-94d8-3512ed8c4fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:01.617070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dda944-19ad-4277-94d8-3512ed8c4fbf service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:01.676283 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 878/1756] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:03.212753 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42903-e49b-4a10-b29b-fed2b63c4880 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:03.213339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42903-e49b-4a10-b29b-fed2b63c4880 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:03.213339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42903-e49b-4a10-b29b-fed2b63c4880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:03.274437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-70f42903-e49b-4a10-b29b-fed2b63c4880 None None] MFA Rules not processed for user `e38a311f2b614a3598f6b51b3a235178`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:03.280319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-70f42903-e49b-4a10-b29b-fed2b63c4880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:03.301532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 879/1757] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Jun 24 23:49:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:03.406181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dda944-19ad-4277-94d8-3512ed8c4fbf service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:03.408826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-01dda944-19ad-4277-94d8-3512ed8c4fbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:03.428360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dda944-19ad-4277-94d8-3512ed8c4fbf service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:03.442629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0de24-c25e-4f5b-bdc4-5e1991ddce39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:03.443601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0de24-c25e-4f5b-bdc4-5e1991ddce39 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:03.444116 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0de24-c25e-4f5b-bdc4-5e1991ddce39 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:03.444545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0de24-c25e-4f5b-bdc4-5e1991ddce39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:03.482190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0de24-c25e-4f5b-bdc4-5e1991ddce39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:03.483940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0de24-c25e-4f5b-bdc4-5e1991ddce39 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:03.517109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 879/1758] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:03.553201 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f42903-e49b-4a10-b29b-fed2b63c4880 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:03.555116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-70f42903-e49b-4a10-b29b-fed2b63c4880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:03.590787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f42903-e49b-4a10-b29b-fed2b63c4880 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:03.618599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:03.619971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:03.620257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:03.620656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] PATH_INFO: `/v3/users/9f18cf838a8e43348afe3d68fa799d20` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:03.622759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f18cf838a8e43348afe3d68fa799d20)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:03.625597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:03.789209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f18cf838a8e43348afe3d68fa799d20'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:03.827859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f18cf838a8e43348afe3d68fa799d20'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:03.842611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f18cf838a8e43348afe3d68fa799d20'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:03.844085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f18cf838a8e43348afe3d68fa799d20'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:03.866499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3d435132-cd0d-4b05-b339-308b732ff180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f18cf838a8e43348afe3d68fa799d20'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:03.919173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 880/1759] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:03 2022] DELETE /identity/v3/users/9f18cf838a8e43348afe3d68fa799d20 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:03.924397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baacf1-7619-49dd-8f80-715c64871a66 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:03.934680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baacf1-7619-49dd-8f80-715c64871a66 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:03.934680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baacf1-7619-49dd-8f80-715c64871a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:03.937028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc0f7-1858-4329-8f9c-d45402900482 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:03.937622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc0f7-1858-4329-8f9c-d45402900482 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:03.938307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc0f7-1858-4329-8f9c-d45402900482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:03.938417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:03.990356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-947dc0f7-1858-4329-8f9c-d45402900482 None None] MFA Rules not processed for user `04e35f840ef045f0b44fab3e57a1a3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:03.997365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-26baacf1-7619-49dd-8f80-715c64871a66 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:03.998314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-947dc0f7-1858-4329-8f9c-d45402900482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:04.007627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-26baacf1-7619-49dd-8f80-715c64871a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:04.164072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 881/1760] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:04.250617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 880/1761] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:04.319339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dc0f7-1858-4329-8f9c-d45402900482 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:04.344221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-947dc0f7-1858-4329-8f9c-d45402900482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:04.420024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:04.530683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dc0f7-1858-4329-8f9c-d45402900482 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:04.536024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eadf2-35ae-476c-9df7-5997f9eef866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:04.536783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eadf2-35ae-476c-9df7-5997f9eef866 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:04.537440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eadf2-35ae-476c-9df7-5997f9eef866 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:04.537440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eadf2-35ae-476c-9df7-5997f9eef866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:04.563200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960eadf2-35ae-476c-9df7-5997f9eef866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:04.565001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960eadf2-35ae-476c-9df7-5997f9eef866 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:04.590926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 882/1762] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:05.092076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baacf1-7619-49dd-8f80-715c64871a66 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:05.093938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-26baacf1-7619-49dd-8f80-715c64871a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:05.130461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baacf1-7619-49dd-8f80-715c64871a66 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:05.152459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:05.153608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:05.153831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:05.154105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] PATH_INFO: `/v3/projects/a4b80b9ac84b47d98df3eab189fb64e8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:05.159886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b80b9ac84b47d98df3eab189fb64e8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:05.162089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:05.315961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b80b9ac84b47d98df3eab189fb64e8', 'request_id': 'req-e2013020-055f-4a7c-a6e0-6941fc58b50b'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.344977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b80b9ac84b47d98df3eab189fb64e8', 'request_id': 'req-e2013020-055f-4a7c-a6e0-6941fc58b50b'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.354791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b80b9ac84b47d98df3eab189fb64e8', 'request_id': 'req-e2013020-055f-4a7c-a6e0-6941fc58b50b'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.383509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] The token cache is being invalidate because project a4b80b9ac84b47d98df3eab189fb64e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:05.383896 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.384656 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.396526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 881/1763] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:05 2022] DELETE /identity/v3/projects/a4b80b9ac84b47d98df3eab189fb64e8 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:05.400690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eadf2-35ae-476c-9df7-5997f9eef866 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:05.419617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eadf2-35ae-476c-9df7-5997f9eef866 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:05.544882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:05.546131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:05.546503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:05.546791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] PATH_INFO: `/v3/users/64cdd10f43e84b76b40f11f9e0026935` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:05.548056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] RBAC: Authorizing `identity:delete_user(user_id=64cdd10f43e84b76b40f11f9e0026935)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:05.550203 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:05.629396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64cdd10f43e84b76b40f11f9e0026935'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.649847 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64cdd10f43e84b76b40f11f9e0026935'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.670726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64cdd10f43e84b76b40f11f9e0026935'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.671503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64cdd10f43e84b76b40f11f9e0026935'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.687019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4f624e45-cd9d-49af-86e4-1f355ad4210a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64cdd10f43e84b76b40f11f9e0026935'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:05.700046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 883/1764] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:05 2022] DELETE /identity/v3/users/64cdd10f43e84b76b40f11f9e0026935 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:06.195551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:06.197315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:06.223008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2013020-055f-4a7c-a6e0-6941fc58b50b admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:06.418489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:06.422170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:06.422474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:06.422762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] PATH_INFO: `/v3/users/a4f97eb8e50e479fbf8403e2096b2867` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:06.426829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4f97eb8e50e479fbf8403e2096b2867)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:06.431089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:06.512818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:06.571224 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f97eb8e50e479fbf8403e2096b2867'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:06.609325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f97eb8e50e479fbf8403e2096b2867'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:06.610123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f97eb8e50e479fbf8403e2096b2867'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:06.610697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f97eb8e50e479fbf8403e2096b2867'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:06.630236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f97eb8e50e479fbf8403e2096b2867'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:06.658334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 882/1765] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:06 2022] DELETE /identity/v3/users/a4f97eb8e50e479fbf8403e2096b2867 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:06.663244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f689420-9ecc-4f9e-b587-ce4fd8b70aa2 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:06.663863 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f689420-9ecc-4f9e-b587-ce4fd8b70aa2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:06.663863 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f689420-9ecc-4f9e-b587-ce4fd8b70aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:06.702494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-5f689420-9ecc-4f9e-b587-ce4fd8b70aa2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:06.705813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5f689420-9ecc-4f9e-b587-ce4fd8b70aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:06.773924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:06.847457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 884/1766] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:07.254285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:07.255783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:07.432519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8c7fc-4f52-4807-8e67-ce695eced865 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:07.444339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7c098-c1a7-4d95-a9e3-9767a53e4d1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:07.446647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7c098-c1a7-4d95-a9e3-9767a53e4d1d service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:07.449679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7c098-c1a7-4d95-a9e3-9767a53e4d1d service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:07.450361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7c098-c1a7-4d95-a9e3-9767a53e4d1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:07.480156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f689420-9ecc-4f9e-b587-ce4fd8b70aa2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:07.481836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5f689420-9ecc-4f9e-b587-ce4fd8b70aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:07.496857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7c098-c1a7-4d95-a9e3-9767a53e4d1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:07.499437 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7c098-c1a7-4d95-a9e3-9767a53e4d1d service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:07.509829 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f689420-9ecc-4f9e-b587-ce4fd8b70aa2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:07.514347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 883/1767] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:07.525781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:07.526728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:07.527065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:07.527304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] PATH_INFO: `/v3/projects/05d1cda6e3904da5824a78fb8defbd8f` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:07.529543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d1cda6e3904da5824a78fb8defbd8f)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:07.544645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:07.729230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d1cda6e3904da5824a78fb8defbd8f', 'request_id': 'req-220e6895-61fa-41ad-aed0-e8358ac34cfc'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:07.751220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d1cda6e3904da5824a78fb8defbd8f', 'request_id': 'req-220e6895-61fa-41ad-aed0-e8358ac34cfc'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:07.796629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d1cda6e3904da5824a78fb8defbd8f', 'request_id': 'req-220e6895-61fa-41ad-aed0-e8358ac34cfc'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:07.833207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] The token cache is being invalidate because project 05d1cda6e3904da5824a78fb8defbd8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:07.834083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:07.835065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:07.843378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 885/1768] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:07 2022] DELETE /identity/v3/projects/05d1cda6e3904da5824a78fb8defbd8f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:07.844466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7c098-c1a7-4d95-a9e3-9767a53e4d1d service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:07.867861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7c098-c1a7-4d95-a9e3-9767a53e4d1d service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:08.072630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:08.076348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:08.076522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:08.076924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] PATH_INFO: `/v3/users/90117311e5b34ac68c112923e7e56f36` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:08.078671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90117311e5b34ac68c112923e7e56f36)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:08.083779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:08.249526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90117311e5b34ac68c112923e7e56f36'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:08.282657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:08.327615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90117311e5b34ac68c112923e7e56f36'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:08.333167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90117311e5b34ac68c112923e7e56f36'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:08.334932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90117311e5b34ac68c112923e7e56f36'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:08.364214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-30b51274-c735-4cce-9480-d8ea3106dc32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90117311e5b34ac68c112923e7e56f36'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:08.433056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 884/1769] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:07 2022] DELETE /identity/v3/users/90117311e5b34ac68c112923e7e56f36 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:09.627477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:09.629644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:09.649651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e6895-61fa-41ad-aed0-e8358ac34cfc admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:09.834342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:09.836248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:09.836561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:09.836854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] PATH_INFO: `/v3/projects/ef17c9b83716491388c4f9cf91106158` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:09.839006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef17c9b83716491388c4f9cf91106158)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:09.841673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:09.985880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef17c9b83716491388c4f9cf91106158', 'request_id': 'req-92a83a45-6356-414f-aebe-4aa50009e057'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.025924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef17c9b83716491388c4f9cf91106158', 'request_id': 'req-92a83a45-6356-414f-aebe-4aa50009e057'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.050991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef17c9b83716491388c4f9cf91106158', 'request_id': 'req-92a83a45-6356-414f-aebe-4aa50009e057'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.080682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:10.150796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] The token cache is being invalidate because project ef17c9b83716491388c4f9cf91106158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:10.153771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.156970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.181272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf01dc-52a8-4986-865f-3865ba3d5a07 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:10.182188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf01dc-52a8-4986-865f-3865ba3d5a07 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:10.182473 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf01dc-52a8-4986-865f-3865ba3d5a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:10.182651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 886/1770] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:09 2022] DELETE /identity/v3/projects/ef17c9b83716491388c4f9cf91106158 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:10.245984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-02cf01dc-52a8-4986-865f-3865ba3d5a07 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:10.247262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-02cf01dc-52a8-4986-865f-3865ba3d5a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:10.264554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:10.266332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:10.452017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 885/1771] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:10.459718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1792e-f2a9-48c5-ab3c-9c64ce0db826 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:10.460316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1792e-f2a9-48c5-ab3c-9c64ce0db826 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:10.460807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1792e-f2a9-48c5-ab3c-9c64ce0db826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:10.537228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6aa1792e-f2a9-48c5-ab3c-9c64ce0db826 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:10.538617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6aa1792e-f2a9-48c5-ab3c-9c64ce0db826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:10.550360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a83a45-6356-414f-aebe-4aa50009e057 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:10.569993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:10.574880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 886/1772] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:10.579322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:10.579691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:10.581836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] PATH_INFO: `/v3/projects/c69cb736720b4f6eb0f1f4ef4bde3131` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:10.589627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69cb736720b4f6eb0f1f4ef4bde3131)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:10.594843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:10.597232 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1792e-f2a9-48c5-ab3c-9c64ce0db826 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:10.634633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1792e-f2a9-48c5-ab3c-9c64ce0db826 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:10.661497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41949c82-0379-4cd9-aae9-3950187963cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:10.662290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41949c82-0379-4cd9-aae9-3950187963cc admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:10.662667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41949c82-0379-4cd9-aae9-3950187963cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:10.663170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41949c82-0379-4cd9-aae9-3950187963cc admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:10.664170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41949c82-0379-4cd9-aae9-3950187963cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:10.664711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41949c82-0379-4cd9-aae9-3950187963cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:10.666547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41949c82-0379-4cd9-aae9-3950187963cc admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:10.721192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 887/1773] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:49:10.731922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41949c82-0379-4cd9-aae9-3950187963cc admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:10.784895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69cb736720b4f6eb0f1f4ef4bde3131', 'request_id': 'req-987567a9-6895-488a-8412-71e17ee88a74'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.791655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41949c82-0379-4cd9-aae9-3950187963cc admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:10.826945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69cb736720b4f6eb0f1f4ef4bde3131', 'request_id': 'req-987567a9-6895-488a-8412-71e17ee88a74'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.858386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69cb736720b4f6eb0f1f4ef4bde3131', 'request_id': 'req-987567a9-6895-488a-8412-71e17ee88a74'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.893702 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] The token cache is being invalidate because project c69cb736720b4f6eb0f1f4ef4bde3131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:10.899554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.901181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-987567a9-6895-488a-8412-71e17ee88a74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:10.909396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 887/1774] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:10 2022] DELETE /identity/v3/projects/c69cb736720b4f6eb0f1f4ef4bde3131 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:10.914343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd10adb-4e2e-4a44-b707-1377b4e66b8b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:10.919919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd10adb-4e2e-4a44-b707-1377b4e66b8b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:10.920498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd10adb-4e2e-4a44-b707-1377b4e66b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:10.974095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-6bd10adb-4e2e-4a44-b707-1377b4e66b8b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:10.975581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6bd10adb-4e2e-4a44-b707-1377b4e66b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:11.021003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252c39b-b73e-4334-bec9-45babe788e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:11.021835 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252c39b-b73e-4334-bec9-45babe788e71 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.023444 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252c39b-b73e-4334-bec9-45babe788e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.023924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252c39b-b73e-4334-bec9-45babe788e71 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.024687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252c39b-b73e-4334-bec9-45babe788e71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:11.027488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252c39b-b73e-4334-bec9-45babe788e71 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:11.114528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 888/1775] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:10 2022] POST /identity/v3/projects => generated 377 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:11.116804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252c39b-b73e-4334-bec9-45babe788e71 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:11.122054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f252c39b-b73e-4334-bec9-45babe788e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:11.137995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 888/1776] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:11.141598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5fa0b-d1eb-4228-baa3-e15b6d875f39 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.142149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5fa0b-d1eb-4228-baa3-e15b6d875f39 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.142655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5fa0b-d1eb-4228-baa3-e15b6d875f39 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.144711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 889/1777] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:49:11.146616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6325c-9435-45f3-9a03-b597e8d70272 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.149725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6325c-9435-45f3-9a03-b597e8d70272 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.150180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6325c-9435-45f3-9a03-b597e8d70272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.206217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-18a6325c-9435-45f3-9a03-b597e8d70272 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:11.208061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-18a6325c-9435-45f3-9a03-b597e8d70272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:11.227180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:11.263970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:11.294658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252c39b-b73e-4334-bec9-45babe788e71 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:11.312206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b0553-7b94-4545-9314-5167d90d5cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:11.313471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b0553-7b94-4545-9314-5167d90d5cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.314101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b0553-7b94-4545-9314-5167d90d5cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.314384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b0553-7b94-4545-9314-5167d90d5cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.315097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b0553-7b94-4545-9314-5167d90d5cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:11.317120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b0553-7b94-4545-9314-5167d90d5cc9 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:11.358733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 889/1778] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:11 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:11.359927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b0553-7b94-4545-9314-5167d90d5cc9 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:11.376065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 890/1779] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:11.382361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b4930-dae6-48a3-864b-77c193858478 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.382630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b4930-dae6-48a3-864b-77c193858478 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.384985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b4930-dae6-48a3-864b-77c193858478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.388380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b0553-7b94-4545-9314-5167d90d5cc9 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:11.407146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a626cbe-5d63-4cad-8aea-7647b1b877c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:11.409937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a626cbe-5d63-4cad-8aea-7647b1b877c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.410176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a626cbe-5d63-4cad-8aea-7647b1b877c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.410462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a626cbe-5d63-4cad-8aea-7647b1b877c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.411232 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a626cbe-5d63-4cad-8aea-7647b1b877c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:11.416645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a626cbe-5d63-4cad-8aea-7647b1b877c3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:11.452477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 890/1780] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:11.454353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-284b4930-dae6-48a3-864b-77c193858478 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:11.456382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-284b4930-dae6-48a3-864b-77c193858478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:11.457308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a626cbe-5d63-4cad-8aea-7647b1b877c3 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:11.503572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a626cbe-5d63-4cad-8aea-7647b1b877c3 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:11.531123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46144375-cbda-4342-b9b0-cdc3ba51059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:11.531821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46144375-cbda-4342-b9b0-cdc3ba51059b admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.532099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46144375-cbda-4342-b9b0-cdc3ba51059b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.532380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46144375-cbda-4342-b9b0-cdc3ba51059b admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.533172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46144375-cbda-4342-b9b0-cdc3ba51059b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:11.533488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46144375-cbda-4342-b9b0-cdc3ba51059b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:11.535130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46144375-cbda-4342-b9b0-cdc3ba51059b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:11.601068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 891/1781] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:49:11.602891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46144375-cbda-4342-b9b0-cdc3ba51059b admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:11.634385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46144375-cbda-4342-b9b0-cdc3ba51059b admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:11.656072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9277a9b-4a8f-4f2a-9572-79979d5bf3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:11.656821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9277a9b-4a8f-4f2a-9572-79979d5bf3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.657117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9277a9b-4a8f-4f2a-9572-79979d5bf3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.657464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9277a9b-4a8f-4f2a-9572-79979d5bf3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.658209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9277a9b-4a8f-4f2a-9572-79979d5bf3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:11.659854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9277a9b-4a8f-4f2a-9572-79979d5bf3f6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:11.689745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 892/1782] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:11.690993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9277a9b-4a8f-4f2a-9572-79979d5bf3f6 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:11.705798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 891/1783] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:49:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:11.706808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b4930-dae6-48a3-864b-77c193858478 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:11.713713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9277a9b-4a8f-4f2a-9572-79979d5bf3f6 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:11.719964 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b4930-dae6-48a3-864b-77c193858478 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:11.724982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9dabe-df9a-469a-87c7-7f2206725987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:11.726818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9dabe-df9a-469a-87c7-7f2206725987 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.731413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9dabe-df9a-469a-87c7-7f2206725987 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.732120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9dabe-df9a-469a-87c7-7f2206725987 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.733058 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9dabe-df9a-469a-87c7-7f2206725987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:11.735350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64fdf5-5f65-4e34-808a-3b81e54f1357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:11.736274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64fdf5-5f65-4e34-808a-3b81e54f1357 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.736558 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64fdf5-5f65-4e34-808a-3b81e54f1357 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.736869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64fdf5-5f65-4e34-808a-3b81e54f1357 admin admin] PATH_INFO: `/v3/projects/73e59ee850944db69fdb9a50588c79fc/users/c18a60a0208b4f88aa8277fea0853d9b/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.741288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9dabe-df9a-469a-87c7-7f2206725987 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:11.777989 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64fdf5-5f65-4e34-808a-3b81e54f1357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e59ee850944db69fdb9a50588c79fc, user_id=c18a60a0208b4f88aa8277fea0853d9b, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:11.780902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64fdf5-5f65-4e34-808a-3b81e54f1357 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:11.800266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 893/1785] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:11 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:11.801391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9dabe-df9a-469a-87c7-7f2206725987 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:11.810252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 892/1785] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:11 2022] PUT /identity/v3/projects/73e59ee850944db69fdb9a50588c79fc/users/c18a60a0208b4f88aa8277fea0853d9b/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:11.811780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca369d-18f0-49f1-b25f-08926425fe4c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.812052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca369d-18f0-49f1-b25f-08926425fe4c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.812323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca369d-18f0-49f1-b25f-08926425fe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.882124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9dabe-df9a-469a-87c7-7f2206725987 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:11.887910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0867e9e-6943-4774-b764-eb83ab868f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:11.888800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0867e9e-6943-4774-b764-eb83ab868f32 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.889198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0867e9e-6943-4774-b764-eb83ab868f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.889590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0867e9e-6943-4774-b764-eb83ab868f32 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.889694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-84ca369d-18f0-49f1-b25f-08926425fe4c None None] MFA Rules not processed for user `c18a60a0208b4f88aa8277fea0853d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:11.890490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0867e9e-6943-4774-b764-eb83ab868f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:11.890886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-84ca369d-18f0-49f1-b25f-08926425fe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:11.894525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0867e9e-6943-4774-b764-eb83ab868f32 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:11.940943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 894/1786] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:11 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:11.942661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633820-6a15-4fb1-a2b6-7c0f4a5756af None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:11.943066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633820-6a15-4fb1-a2b6-7c0f4a5756af None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:11.945338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38633820-6a15-4fb1-a2b6-7c0f4a5756af None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:11.948740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 895/1787] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:49:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:49:11.950305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38633820-6a15-4fb1-a2b6-7c0f4a5756af None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:11.981044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38633820-6a15-4fb1-a2b6-7c0f4a5756af None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:12.001953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f02d8-8b39-422a-b6eb-c8e6cd7ddc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:12.003843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f02d8-8b39-422a-b6eb-c8e6cd7ddc5f admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.004129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f02d8-8b39-422a-b6eb-c8e6cd7ddc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.004411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f02d8-8b39-422a-b6eb-c8e6cd7ddc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.005181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f02d8-8b39-422a-b6eb-c8e6cd7ddc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:12.006016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 893/1788] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:12.012920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca369d-18f0-49f1-b25f-08926425fe4c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.013134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f02d8-8b39-422a-b6eb-c8e6cd7ddc5f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:12.045694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 896/1789] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:12.050003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f02d8-8b39-422a-b6eb-c8e6cd7ddc5f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.063278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca369d-18f0-49f1-b25f-08926425fe4c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:12.065657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f02d8-8b39-422a-b6eb-c8e6cd7ddc5f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:12.078519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450517cb-6b83-42ab-831a-31f3bba210ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:12.097412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450517cb-6b83-42ab-831a-31f3bba210ef admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.098419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450517cb-6b83-42ab-831a-31f3bba210ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.100245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450517cb-6b83-42ab-831a-31f3bba210ef admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.102285 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0630bc-584f-45f7-a4aa-1945c51627a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:12.102617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450517cb-6b83-42ab-831a-31f3bba210ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:12.108331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0630bc-584f-45f7-a4aa-1945c51627a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.109002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0630bc-584f-45f7-a4aa-1945c51627a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.109463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0630bc-584f-45f7-a4aa-1945c51627a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.112066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0630bc-584f-45f7-a4aa-1945c51627a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:12.112807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450517cb-6b83-42ab-831a-31f3bba210ef admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:12.129423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0630bc-584f-45f7-a4aa-1945c51627a5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:12.186561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 897/1790] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:49:12.192193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d717e77-55df-4fbf-b466-f3d175da6fd8 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.192470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d717e77-55df-4fbf-b466-f3d175da6fd8 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.192781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d717e77-55df-4fbf-b466-f3d175da6fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.232187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 894/1791] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:12 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:12.233123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450517cb-6b83-42ab-831a-31f3bba210ef admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.235502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6d717e77-55df-4fbf-b466-f3d175da6fd8 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:12.236800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6d717e77-55df-4fbf-b466-f3d175da6fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:12.244900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:12.319218 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:12.404190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450517cb-6b83-42ab-831a-31f3bba210ef admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:12.412128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 898/1792] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:49:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:12.413688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d717e77-55df-4fbf-b466-f3d175da6fd8 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.414944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c29cf2-7c27-4170-b7f0-a31f5e6642e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:12.416070 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c29cf2-7c27-4170-b7f0-a31f5e6642e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.416356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c29cf2-7c27-4170-b7f0-a31f5e6642e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.416658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c29cf2-7c27-4170-b7f0-a31f5e6642e0 admin admin] PATH_INFO: `/v3/projects/7e652bdfeb0d42358829e18ac7dcfe03/users/5bf4c1e195444a85922b046a42fdac06/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.432563 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d717e77-55df-4fbf-b466-f3d175da6fd8 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:12.451139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ae9b-1713-4e34-b400-b909dba7758d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:12.451870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ae9b-1713-4e34-b400-b909dba7758d admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.452167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ae9b-1713-4e34-b400-b909dba7758d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.452450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ae9b-1713-4e34-b400-b909dba7758d admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.453195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ae9b-1713-4e34-b400-b909dba7758d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:12.455494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ae9b-1713-4e34-b400-b909dba7758d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:12.459786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c29cf2-7c27-4170-b7f0-a31f5e6642e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e652bdfeb0d42358829e18ac7dcfe03, user_id=5bf4c1e195444a85922b046a42fdac06, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:12.467296 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c29cf2-7c27-4170-b7f0-a31f5e6642e0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:12.491733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 895/1793] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:12 2022] PUT /identity/v3/projects/7e652bdfeb0d42358829e18ac7dcfe03/users/5bf4c1e195444a85922b046a42fdac06/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:12.505394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a58001-3fd1-405f-bbd4-77dd95e8f19c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.505726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a58001-3fd1-405f-bbd4-77dd95e8f19c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.505961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a58001-3fd1-405f-bbd4-77dd95e8f19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.508566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 899/1794] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:12 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:12.512237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ae9b-1713-4e34-b400-b909dba7758d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.533064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ae9b-1713-4e34-b400-b909dba7758d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:12.567350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-e6a58001-3fd1-405f-bbd4-77dd95e8f19c None None] MFA Rules not processed for user `5bf4c1e195444a85922b046a42fdac06`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:12.570649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e6a58001-3fd1-405f-bbd4-77dd95e8f19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:12.628290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ef7be-08ca-4ec3-b1c8-2766dad0a48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:12.629348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ef7be-08ca-4ec3-b1c8-2766dad0a48d admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.629647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ef7be-08ca-4ec3-b1c8-2766dad0a48d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.629933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ef7be-08ca-4ec3-b1c8-2766dad0a48d admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.630859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ef7be-08ca-4ec3-b1c8-2766dad0a48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:12.632598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ef7be-08ca-4ec3-b1c8-2766dad0a48d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:12.649723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 900/1795] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:49:12.653900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ef7be-08ca-4ec3-b1c8-2766dad0a48d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.683396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ef7be-08ca-4ec3-b1c8-2766dad0a48d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:12.696334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779469a8-fcb8-4c61-9d2b-b0235ba17a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:12.697655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779469a8-fcb8-4c61-9d2b-b0235ba17a66 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.699700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779469a8-fcb8-4c61-9d2b-b0235ba17a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.700127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779469a8-fcb8-4c61-9d2b-b0235ba17a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.703033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779469a8-fcb8-4c61-9d2b-b0235ba17a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:12.706126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779469a8-fcb8-4c61-9d2b-b0235ba17a66 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:12.737571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 896/1796] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:12.742329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a58001-3fd1-405f-bbd4-77dd95e8f19c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.763805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 901/1797] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:12.765072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779469a8-fcb8-4c61-9d2b-b0235ba17a66 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.766654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a58001-3fd1-405f-bbd4-77dd95e8f19c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:12.783686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cffbb-729a-445a-a5fd-33bf93a66177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:12.784942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cffbb-729a-445a-a5fd-33bf93a66177 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.785235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cffbb-729a-445a-a5fd-33bf93a66177 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.785516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cffbb-729a-445a-a5fd-33bf93a66177 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.786313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cffbb-729a-445a-a5fd-33bf93a66177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:12.788438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cffbb-729a-445a-a5fd-33bf93a66177 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:12.795451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779469a8-fcb8-4c61-9d2b-b0235ba17a66 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:12.809449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dbc3f-fc0c-4cc7-91a9-e61120bee70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:12.810840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dbc3f-fc0c-4cc7-91a9-e61120bee70c admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:12.811327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dbc3f-fc0c-4cc7-91a9-e61120bee70c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:12.811689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dbc3f-fc0c-4cc7-91a9-e61120bee70c admin admin] PATH_INFO: `/v3/projects/4baa775b0dbe401e94bdafc44c4fcb55/users/670a63ff653d4b9abc9c5d438e6f01e7/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:12.855624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 897/1798] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:12 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:12.857963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cffbb-729a-445a-a5fd-33bf93a66177 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.862437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-413cffbb-729a-445a-a5fd-33bf93a66177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:12.864279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dbc3f-fc0c-4cc7-91a9-e61120bee70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4baa775b0dbe401e94bdafc44c4fcb55, user_id=670a63ff653d4b9abc9c5d438e6f01e7, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:12.867584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dbc3f-fc0c-4cc7-91a9-e61120bee70c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:12.892898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 902/1799] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:12 2022] PUT /identity/v3/projects/4baa775b0dbe401e94bdafc44c4fcb55/users/670a63ff653d4b9abc9c5d438e6f01e7/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:12.894828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dbc3f-fc0c-4cc7-91a9-e61120bee70c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:12.971060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:13.014572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cffbb-729a-445a-a5fd-33bf93a66177 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:13.195620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5911fa-b1b7-4ef4-aeeb-9fab920b9b05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:13.197456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5911fa-b1b7-4ef4-aeeb-9fab920b9b05 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:13.198696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5911fa-b1b7-4ef4-aeeb-9fab920b9b05 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:13.199942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5911fa-b1b7-4ef4-aeeb-9fab920b9b05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:13.225380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5911fa-b1b7-4ef4-aeeb-9fab920b9b05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:13.230469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5911fa-b1b7-4ef4-aeeb-9fab920b9b05 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:13.265266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 898/1800] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:13.267954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81040d60-d222-4d2c-88ff-6a3e321e00d6 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:13.268868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81040d60-d222-4d2c-88ff-6a3e321e00d6 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:13.269697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81040d60-d222-4d2c-88ff-6a3e321e00d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:13.315029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dbc3f-fc0c-4cc7-91a9-e61120bee70c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:13.322393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfa716-dc63-476e-ac94-49a609918622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:13.324251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfa716-dc63-476e-ac94-49a609918622 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:13.324789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfa716-dc63-476e-ac94-49a609918622 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:13.325500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfa716-dc63-476e-ac94-49a609918622 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:13.327203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfa716-dc63-476e-ac94-49a609918622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:13.331129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfa716-dc63-476e-ac94-49a609918622 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:13.342533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-81040d60-d222-4d2c-88ff-6a3e321e00d6 None None] MFA Rules not processed for user `670a63ff653d4b9abc9c5d438e6f01e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:13.347847 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-81040d60-d222-4d2c-88ff-6a3e321e00d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:13.364468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:13.383873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 903/1801] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:12 2022] POST /identity/v3/users => generated 297 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:13.389291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfa716-dc63-476e-ac94-49a609918622 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:13.422643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfa716-dc63-476e-ac94-49a609918622 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:13.438243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fd992-d958-43ba-a3af-719bfa3a8f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:13.439006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fd992-d958-43ba-a3af-719bfa3a8f04 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:13.439321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fd992-d958-43ba-a3af-719bfa3a8f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:13.439730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fd992-d958-43ba-a3af-719bfa3a8f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:13.440609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fd992-d958-43ba-a3af-719bfa3a8f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:13.448208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fd992-d958-43ba-a3af-719bfa3a8f04 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:13.486195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 904/1802] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:13.486195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fd992-d958-43ba-a3af-719bfa3a8f04 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:13.505350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fd992-d958-43ba-a3af-719bfa3a8f04 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:13.519478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca108c-1451-4471-9f27-0f7dda84d5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:13.522236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca108c-1451-4471-9f27-0f7dda84d5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:13.523842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca108c-1451-4471-9f27-0f7dda84d5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:13.524246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca108c-1451-4471-9f27-0f7dda84d5d1 admin admin] PATH_INFO: `/v3/projects/807a586c2ca24378b7199be9f51c9136/users/5d9e35b17c9e49ac86a89afe1ef9ca9a/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:13.545244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 899/1803] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:13.549193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81040d60-d222-4d2c-88ff-6a3e321e00d6 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:13.565463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca108c-1451-4471-9f27-0f7dda84d5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=807a586c2ca24378b7199be9f51c9136, user_id=5d9e35b17c9e49ac86a89afe1ef9ca9a, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:13.571223 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca108c-1451-4471-9f27-0f7dda84d5d1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:13.576449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81040d60-d222-4d2c-88ff-6a3e321e00d6 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:13.595505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a1bd6-6bfc-46b6-ab3a-5c183acd506b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:13.598680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a1bd6-6bfc-46b6-ab3a-5c183acd506b admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:13.599539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a1bd6-6bfc-46b6-ab3a-5c183acd506b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:13.600701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a1bd6-6bfc-46b6-ab3a-5c183acd506b admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:13.602953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a1bd6-6bfc-46b6-ab3a-5c183acd506b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:13.606637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a1bd6-6bfc-46b6-ab3a-5c183acd506b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:13.609389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 905/1804] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:13 2022] PUT /identity/v3/projects/807a586c2ca24378b7199be9f51c9136/users/5d9e35b17c9e49ac86a89afe1ef9ca9a/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:13.610472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca108c-1451-4471-9f27-0f7dda84d5d1 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:13.614419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-69ca108c-1451-4471-9f27-0f7dda84d5d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:13.673801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 900/1805] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:13 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:13.675529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a1bd6-6bfc-46b6-ab3a-5c183acd506b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:13.778838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:13.799379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca108c-1451-4471-9f27-0f7dda84d5d1 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:13.804786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:13.805645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:13.805923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:13.806207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] PATH_INFO: `/v3/projects/cee9fb8005f140feb75154f368f19b90` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:13.808400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee9fb8005f140feb75154f368f19b90)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:13.811053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:13.817192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a1bd6-6bfc-46b6-ab3a-5c183acd506b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:13.845204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a4fc2-3980-4e06-bbd6-33ef19a8caab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:13.845917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4fc2-3980-4e06-bbd6-33ef19a8caab admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:13.846546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4fc2-3980-4e06-bbd6-33ef19a8caab admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:13.849714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4fc2-3980-4e06-bbd6-33ef19a8caab admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:13.850525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a4fc2-3980-4e06-bbd6-33ef19a8caab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:13.854204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a4fc2-3980-4e06-bbd6-33ef19a8caab admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:13.874448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 901/1806] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:13.882993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a4fc2-3980-4e06-bbd6-33ef19a8caab admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:13.901987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a4fc2-3980-4e06-bbd6-33ef19a8caab admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:13.915920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee9fb8005f140feb75154f368f19b90', 'request_id': 'req-c24c62bc-fb83-4419-808e-4a2b1da0888b'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:13.933713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee9fb8005f140feb75154f368f19b90', 'request_id': 'req-c24c62bc-fb83-4419-808e-4a2b1da0888b'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:13.945340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee9fb8005f140feb75154f368f19b90', 'request_id': 'req-c24c62bc-fb83-4419-808e-4a2b1da0888b'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:13.967258 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] The token cache is being invalidate because project cee9fb8005f140feb75154f368f19b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:13.968097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:13.968866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:13.985676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 906/1807] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:13 2022] DELETE /identity/v3/projects/cee9fb8005f140feb75154f368f19b90 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:13.986749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:13.994352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:14.025308 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562c056-2492-45ed-a598-73c8986ffd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:14.026937 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562c056-2492-45ed-a598-73c8986ffd4c admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.028031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562c056-2492-45ed-a598-73c8986ffd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.028661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562c056-2492-45ed-a598-73c8986ffd4c admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.029881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562c056-2492-45ed-a598-73c8986ffd4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:14.032658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562c056-2492-45ed-a598-73c8986ffd4c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:14.063077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 902/1808] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:13 2022] POST /identity/v3/users => generated 295 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:14.067254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562c056-2492-45ed-a598-73c8986ffd4c admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:14.070471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c562c056-2492-45ed-a598-73c8986ffd4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:14.084454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562c056-2492-45ed-a598-73c8986ffd4c admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:14.090666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c5327-14f6-446c-adb7-0ac48980db52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:14.091833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c5327-14f6-446c-adb7-0ac48980db52 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.092232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c5327-14f6-446c-adb7-0ac48980db52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.092665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c5327-14f6-446c-adb7-0ac48980db52 admin admin] PATH_INFO: `/v3/projects/807a586c2ca24378b7199be9f51c9136/users/5d9e35b17c9e49ac86a89afe1ef9ca9a/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.094986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c5327-14f6-446c-adb7-0ac48980db52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=807a586c2ca24378b7199be9f51c9136, user_id=5d9e35b17c9e49ac86a89afe1ef9ca9a, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:14.098613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c5327-14f6-446c-adb7-0ac48980db52 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:14.112238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:14.115392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 903/1809] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:14 2022] PUT /identity/v3/projects/807a586c2ca24378b7199be9f51c9136/users/5d9e35b17c9e49ac86a89afe1ef9ca9a/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:14.116562 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c5327-14f6-446c-adb7-0ac48980db52 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:14.128108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d30c5327-14f6-446c-adb7-0ac48980db52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:14.165216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c62bc-fb83-4419-808e-4a2b1da0888b admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:14.219420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c5327-14f6-446c-adb7-0ac48980db52 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:14.224847 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:14.226015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.226479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.226940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] PATH_INFO: `/v3/users/ad79487e8f77428a80613d9847e8f1cd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.228360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad79487e8f77428a80613d9847e8f1cd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:14.230603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:14.263134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:14.264630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.265609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.265890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.269026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:14.288576 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:14.290623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:14.303381 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad79487e8f77428a80613d9847e8f1cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:14.305121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 907/1810] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:14.305970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:14.307433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:14.316253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:14.323639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947e4af-3772-4794-963a-803615c358f7 service nova] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:14.324530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad79487e8f77428a80613d9847e8f1cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:14.337714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b2cd9-fed5-462a-a7c8-bbbd64fb7888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:14.338408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b2cd9-fed5-462a-a7c8-bbbd64fb7888 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.338660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b2cd9-fed5-462a-a7c8-bbbd64fb7888 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.341786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad79487e8f77428a80613d9847e8f1cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:14.342590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b2cd9-fed5-462a-a7c8-bbbd64fb7888 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.343208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad79487e8f77428a80613d9847e8f1cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:14.343412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b2cd9-fed5-462a-a7c8-bbbd64fb7888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:14.345254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b2cd9-fed5-462a-a7c8-bbbd64fb7888 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:14.360802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5158ba91-19af-4c51-87ce-3d3565e34e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad79487e8f77428a80613d9847e8f1cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:14.363717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 904/1811] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:14 2022] DELETE /identity/v3/users/ad79487e8f77428a80613d9847e8f1cd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:14.365156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3c17-9902-4b7d-abf4-de5693a1abf2 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.365454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3c17-9902-4b7d-abf4-de5693a1abf2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.365739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3c17-9902-4b7d-abf4-de5693a1abf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.373269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 908/1812] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:14.374380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b2cd9-fed5-462a-a7c8-bbbd64fb7888 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:14.377029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0b7b2cd9-fed5-462a-a7c8-bbbd64fb7888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:14.400437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-ff2c3c17-9902-4b7d-abf4-de5693a1abf2 None None] MFA Rules not processed for user `5d9e35b17c9e49ac86a89afe1ef9ca9a`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:14.402588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ff2c3c17-9902-4b7d-abf4-de5693a1abf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:14.407347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b2cd9-fed5-462a-a7c8-bbbd64fb7888 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:14.501119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 905/1813] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:14.516513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c3c17-9902-4b7d-abf4-de5693a1abf2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:14.519076 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ff2c3c17-9902-4b7d-abf4-de5693a1abf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:14.521231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeffef3-48df-469f-9bff-d0564c4af0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:14.522778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeffef3-48df-469f-9bff-d0564c4af0fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.523148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeffef3-48df-469f-9bff-d0564c4af0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.523622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeffef3-48df-469f-9bff-d0564c4af0fc admin admin] PATH_INFO: `/v3/projects/77d38b902ca044b5ba0ee012128da294/users/1889b65a873e41528054e735df8f7e86/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.540955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeffef3-48df-469f-9bff-d0564c4af0fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d38b902ca044b5ba0ee012128da294, user_id=1889b65a873e41528054e735df8f7e86, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:14.543987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeffef3-48df-469f-9bff-d0564c4af0fc admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:14.567876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 909/1814] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:14 2022] PUT /identity/v3/projects/77d38b902ca044b5ba0ee012128da294/users/1889b65a873e41528054e735df8f7e86/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:14.569168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeffef3-48df-469f-9bff-d0564c4af0fc admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:14.620090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c3c17-9902-4b7d-abf4-de5693a1abf2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:14.645169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:14.673745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeffef3-48df-469f-9bff-d0564c4af0fc admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:14.679956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6e03c-f901-4498-bc0c-b93d71c6da14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:14.680920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6e03c-f901-4498-bc0c-b93d71c6da14 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.681496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6e03c-f901-4498-bc0c-b93d71c6da14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.681622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6e03c-f901-4498-bc0c-b93d71c6da14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.683759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6e03c-f901-4498-bc0c-b93d71c6da14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:14.685781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6e03c-f901-4498-bc0c-b93d71c6da14 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:14.696545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c9491-2cb4-4f0b-b7bf-38daf9eb987d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:14.698319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c9491-2cb4-4f0b-b7bf-38daf9eb987d service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.698785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c9491-2cb4-4f0b-b7bf-38daf9eb987d service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.699391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c9491-2cb4-4f0b-b7bf-38daf9eb987d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.705342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 910/1815] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:14.706880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6e03c-f901-4498-bc0c-b93d71c6da14 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:14.720854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6e03c-f901-4498-bc0c-b93d71c6da14 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:14.730446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e322e3-7496-460d-bf8f-36e880baae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:14.731849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e322e3-7496-460d-bf8f-36e880baae7b admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.732226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e322e3-7496-460d-bf8f-36e880baae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.732668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e322e3-7496-460d-bf8f-36e880baae7b admin admin] PATH_INFO: `/v3/projects/77d38b902ca044b5ba0ee012128da294/users/1889b65a873e41528054e735df8f7e86/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.735877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e322e3-7496-460d-bf8f-36e880baae7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d38b902ca044b5ba0ee012128da294, user_id=1889b65a873e41528054e735df8f7e86, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:14.739165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e322e3-7496-460d-bf8f-36e880baae7b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:14.750008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:14.785904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 911/1816] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:14 2022] PUT /identity/v3/projects/77d38b902ca044b5ba0ee012128da294/users/1889b65a873e41528054e735df8f7e86/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:14.787888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6e7a4-ab50-4ff7-9556-7ad13062d017 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:14.788185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6e7a4-ab50-4ff7-9556-7ad13062d017 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:14.788463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6e7a4-ab50-4ff7-9556-7ad13062d017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:14.828996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2ff6e7a4-ab50-4ff7-9556-7ad13062d017 None None] MFA Rules not processed for user `1889b65a873e41528054e735df8f7e86`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:14.830930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2ff6e7a4-ab50-4ff7-9556-7ad13062d017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:14.861193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c9491-2cb4-4f0b-b7bf-38daf9eb987d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:14.864188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c9491-2cb4-4f0b-b7bf-38daf9eb987d service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:14.892115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 906/1817] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:14.956543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 912/1818] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:14.964409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c9491-2cb4-4f0b-b7bf-38daf9eb987d service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:14.980901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c9491-2cb4-4f0b-b7bf-38daf9eb987d service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:15.111013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c685e8-a54d-45a2-baae-e025c714429a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:15.112128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c685e8-a54d-45a2-baae-e025c714429a service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:15.112882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c685e8-a54d-45a2-baae-e025c714429a service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:15.113375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c685e8-a54d-45a2-baae-e025c714429a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:15.130516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c685e8-a54d-45a2-baae-e025c714429a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:15.132003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c685e8-a54d-45a2-baae-e025c714429a service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:15.173970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 907/1819] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:15.303849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6e7a4-ab50-4ff7-9556-7ad13062d017 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:15.326739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6e7a4-ab50-4ff7-9556-7ad13062d017 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:15.333774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:15.334743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:15.335188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:15.335618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:15.338762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:15.432954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:15.435279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:15.475578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 913/1820] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:15.590682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c685e8-a54d-45a2-baae-e025c714429a service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:15.698848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:15.700673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:15.721591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c685e8-a54d-45a2-baae-e025c714429a service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:15.736411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225fa5-752d-4823-b77a-f0f7ab083dda tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d9e35b17c9e49ac86a89afe1ef9ca9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '807a586c2ca24378b7199be9f51c9136', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:15.737610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225fa5-752d-4823-b77a-f0f7ab083dda tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:15.739524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225fa5-752d-4823-b77a-f0f7ab083dda tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:15.739960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c225fa5-752d-4823-b77a-f0f7ab083dda tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:15.741302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c225fa5-752d-4823-b77a-f0f7ab083dda tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:15.741814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c225fa5-752d-4823-b77a-f0f7ab083dda tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:15.744923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c225fa5-752d-4823-b77a-f0f7ab083dda tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:15.794658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 908/1821] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:49:15.809775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225fa5-752d-4823-b77a-f0f7ab083dda tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:15.831040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c225fa5-752d-4823-b77a-f0f7ab083dda tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:15.845460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:15.854413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ca17b-5645-4697-9f21-5379b251969c tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d9e35b17c9e49ac86a89afe1ef9ca9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '807a586c2ca24378b7199be9f51c9136', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:15.855222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ca17b-5645-4697-9f21-5379b251969c tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:15.855474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ca17b-5645-4697-9f21-5379b251969c tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:15.855754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ca17b-5645-4697-9f21-5379b251969c tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:15.856463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ca17b-5645-4697-9f21-5379b251969c tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:15.858652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ca17b-5645-4697-9f21-5379b251969c tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:15.861743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcab520-e5b1-4e1a-8a24-36328813614d service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:15.869403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:15.872437 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:15.873687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:15.877092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] PATH_INFO: `/v3/projects/69b0880e11ec44cabce1e7f2dde601ba` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:15.884050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b0880e11ec44cabce1e7f2dde601ba)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:15.888190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:15.946684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 909/1822] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:15 2022] POST /identity/v3/projects => generated 353 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:15.951428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ca17b-5645-4697-9f21-5379b251969c tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:16.018195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b0880e11ec44cabce1e7f2dde601ba', 'request_id': 'req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.044409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b0880e11ec44cabce1e7f2dde601ba', 'request_id': 'req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.054355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b0880e11ec44cabce1e7f2dde601ba', 'request_id': 'req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.059378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:16.087305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] The token cache is being invalidate because project 69b0880e11ec44cabce1e7f2dde601ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:16.087535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.090452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fc9e7aa0-fb6c-4c19-b84b-58a3fcbc286f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.092068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ca17b-5645-4697-9f21-5379b251969c tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:16.102755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 914/1823] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:15 2022] DELETE /identity/v3/projects/69b0880e11ec44cabce1e7f2dde601ba => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:16.105555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a18b2-64fc-41ec-81c7-f7b5bc4b107e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:16.105924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a18b2-64fc-41ec-81c7-f7b5bc4b107e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:16.106275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a18b2-64fc-41ec-81c7-f7b5bc4b107e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:16.154541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-8f4a18b2-64fc-41ec-81c7-f7b5bc4b107e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:16.159004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8f4a18b2-64fc-41ec-81c7-f7b5bc4b107e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:16.367799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 915/1824] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:16.377517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54695680-7299-449f-b073-a580df9e18ba None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:16.377854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54695680-7299-449f-b073-a580df9e18ba None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:16.378143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54695680-7299-449f-b073-a580df9e18ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:16.417467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-54695680-7299-449f-b073-a580df9e18ba None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:16.419424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-54695680-7299-449f-b073-a580df9e18ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:16.425456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6947c8-8799-40fa-a730-4d70c9effbdb tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d9e35b17c9e49ac86a89afe1ef9ca9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '807a586c2ca24378b7199be9f51c9136', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:16.426149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6947c8-8799-40fa-a730-4d70c9effbdb tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:16.427461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6947c8-8799-40fa-a730-4d70c9effbdb tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:16.427758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6947c8-8799-40fa-a730-4d70c9effbdb tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:16.429173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6947c8-8799-40fa-a730-4d70c9effbdb tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:16.429497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6947c8-8799-40fa-a730-4d70c9effbdb tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:16.433356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6947c8-8799-40fa-a730-4d70c9effbdb tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:16.441597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 916/1825] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:16.445910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54695680-7299-449f-b073-a580df9e18ba None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:16.461256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54695680-7299-449f-b073-a580df9e18ba None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:16.462219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 910/1826] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 511 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:49:16.467045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:16.469311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:16.469311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:16.469311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:16.469311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:16.470013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:16.480980 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:16.514740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 917/1827] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:49:16.515844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6947c8-8799-40fa-a730-4d70c9effbdb tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:16.536266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:16.538269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:16.538575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6947c8-8799-40fa-a730-4d70c9effbdb tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:16.550966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83981a26-a842-424c-9cbf-e74ee9722e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:16.551772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83981a26-a842-424c-9cbf-e74ee9722e21 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:16.552064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83981a26-a842-424c-9cbf-e74ee9722e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:16.552342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83981a26-a842-424c-9cbf-e74ee9722e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:16.553097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83981a26-a842-424c-9cbf-e74ee9722e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:16.558428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83981a26-a842-424c-9cbf-e74ee9722e21 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:16.561601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7c909-7f3f-412c-9c4f-e72770dcc652 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:16.571271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d9e35b17c9e49ac86a89afe1ef9ca9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '807a586c2ca24378b7199be9f51c9136', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:16.572047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:16.572341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:16.572699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] PATH_INFO: `/v3/projects/6c307ef8767c451ab831f0a2c76b85d6` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:16.575472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6c307ef8767c451ab831f0a2c76b85d6)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:16.577556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:16.628560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 911/1828] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:16 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:16.634742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83981a26-a842-424c-9cbf-e74ee9722e21 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:16.663451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c307ef8767c451ab831f0a2c76b85d6', 'request_id': 'req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.723700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c307ef8767c451ab831f0a2c76b85d6', 'request_id': 'req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.746172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c307ef8767c451ab831f0a2c76b85d6', 'request_id': 'req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.751471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:16.783856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] The token cache is being invalidate because project 6c307ef8767c451ab831f0a2c76b85d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:16.784359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.786842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:16.788297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83981a26-a842-424c-9cbf-e74ee9722e21 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:16.790910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 918/1829] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:16 2022] DELETE /identity/v3/projects/6c307ef8767c451ab831f0a2c76b85d6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:16.869356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586a3c-234b-45d7-a64a-2bc7389ac872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:16.870627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586a3c-234b-45d7-a64a-2bc7389ac872 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:16.871169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586a3c-234b-45d7-a64a-2bc7389ac872 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:16.872001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d586a3c-234b-45d7-a64a-2bc7389ac872 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:16.872881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d586a3c-234b-45d7-a64a-2bc7389ac872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:16.876772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d586a3c-234b-45d7-a64a-2bc7389ac872 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:16.896542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:16.910223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 912/1830] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:16 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:16.914452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:16.916037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:16.936378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76471f22-d5b6-49fe-b80c-f3f57c2a7b4e tempest-VolumeQuotaClassesTest-1430460251 tempest-VolumeQuotaClassesTest-1430460251-project-admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:16.945631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb68206-eecd-47d1-967b-0185c9b6b082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:16.946329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb68206-eecd-47d1-967b-0185c9b6b082 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:16.959401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb68206-eecd-47d1-967b-0185c9b6b082 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:16.959401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb68206-eecd-47d1-967b-0185c9b6b082 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:16.959401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb68206-eecd-47d1-967b-0185c9b6b082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:16.960485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb68206-eecd-47d1-967b-0185c9b6b082 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:16.975944 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 919/1831] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:16.981109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586a3c-234b-45d7-a64a-2bc7389ac872 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:16.996747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d586a3c-234b-45d7-a64a-2bc7389ac872 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:17.009880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef58706-76da-443f-929d-56514aaf0777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:17.011389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef58706-76da-443f-929d-56514aaf0777 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:17.011678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef58706-76da-443f-929d-56514aaf0777 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:17.012375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef58706-76da-443f-929d-56514aaf0777 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:17.014141 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef58706-76da-443f-929d-56514aaf0777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:17.015782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef58706-76da-443f-929d-56514aaf0777 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:17.032552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 913/1832] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:49:17.033967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb68206-eecd-47d1-967b-0185c9b6b082 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:17.047530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb68206-eecd-47d1-967b-0185c9b6b082 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:17.053193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07df29-38a8-4a63-81bc-e19a3cb332a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:17.054098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07df29-38a8-4a63-81bc-e19a3cb332a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:17.054384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07df29-38a8-4a63-81bc-e19a3cb332a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:17.054664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07df29-38a8-4a63-81bc-e19a3cb332a6 admin admin] PATH_INFO: `/v3/projects/b5c1794ea4ab49fda088b08b2f63d03e/users/0e39a6172b324166874d5086746b1e18/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:17.075575 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07df29-38a8-4a63-81bc-e19a3cb332a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c1794ea4ab49fda088b08b2f63d03e, user_id=0e39a6172b324166874d5086746b1e18, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:17.079856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07df29-38a8-4a63-81bc-e19a3cb332a6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:17.092040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef58706-76da-443f-929d-56514aaf0777 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:17.094274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-aef58706-76da-443f-929d-56514aaf0777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:17.102084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 920/1833] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:17 2022] PUT /identity/v3/projects/b5c1794ea4ab49fda088b08b2f63d03e/users/0e39a6172b324166874d5086746b1e18/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:17.113778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c807027-cd14-4363-b00f-4d48e41ca9bb None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:17.114103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c807027-cd14-4363-b00f-4d48e41ca9bb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:17.114353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c807027-cd14-4363-b00f-4d48e41ca9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:17.116880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef58706-76da-443f-929d-56514aaf0777 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:17.189566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6c807027-cd14-4363-b00f-4d48e41ca9bb None None] MFA Rules not processed for user `0e39a6172b324166874d5086746b1e18`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:17.190847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6c807027-cd14-4363-b00f-4d48e41ca9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:17.200334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:17.202220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:17.202723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:17.203339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] PATH_INFO: `/v3/users/5bf4c1e195444a85922b046a42fdac06` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:17.206246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf4c1e195444a85922b046a42fdac06)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:17.212383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:17.321126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf4c1e195444a85922b046a42fdac06'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:17.343645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf4c1e195444a85922b046a42fdac06'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:17.364748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf4c1e195444a85922b046a42fdac06'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:17.375018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf4c1e195444a85922b046a42fdac06'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:17.381779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1d22b452-7b26-429b-9b17-46e864ce96fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf4c1e195444a85922b046a42fdac06'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:17.386449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 921/1834] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:17.391485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c807027-cd14-4363-b00f-4d48e41ca9bb None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:17.406977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 914/1835] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:17 2022] DELETE /identity/v3/users/5bf4c1e195444a85922b046a42fdac06 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:17.409985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1afca-b49d-4112-9a0f-caff805ccc46 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:17.410547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1afca-b49d-4112-9a0f-caff805ccc46 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:17.410913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1afca-b49d-4112-9a0f-caff805ccc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:17.424447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c807027-cd14-4363-b00f-4d48e41ca9bb None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:17.437639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-92e1afca-b49d-4112-9a0f-caff805ccc46 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:17.439591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-92e1afca-b49d-4112-9a0f-caff805ccc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:17.630838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 915/1836] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:17.634949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02f540-a235-4af5-a60c-038978cdb854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:17.635707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02f540-a235-4af5-a60c-038978cdb854 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:17.636062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02f540-a235-4af5-a60c-038978cdb854 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:17.636717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02f540-a235-4af5-a60c-038978cdb854 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:17.637113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02f540-a235-4af5-a60c-038978cdb854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:17.639156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02f540-a235-4af5-a60c-038978cdb854 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:17.694720 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1afca-b49d-4112-9a0f-caff805ccc46 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:17.696474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-92e1afca-b49d-4112-9a0f-caff805ccc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:17.711982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:17.714860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 922/1837] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:17 2022] POST /identity/v3/projects => generated 375 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:17.721626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02f540-a235-4af5-a60c-038978cdb854 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:17.738039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6c02f540-a235-4af5-a60c-038978cdb854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:17.825242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:17.881359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02f540-a235-4af5-a60c-038978cdb854 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:17.884774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1afca-b49d-4112-9a0f-caff805ccc46 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:17.888461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf582a-e86d-4d42-8430-ad47e7d2e0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:17.889464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf582a-e86d-4d42-8430-ad47e7d2e0ee admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:17.889885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf582a-e86d-4d42-8430-ad47e7d2e0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:17.890312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf582a-e86d-4d42-8430-ad47e7d2e0ee admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:17.891339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf582a-e86d-4d42-8430-ad47e7d2e0ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:17.894191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf582a-e86d-4d42-8430-ad47e7d2e0ee admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:17.922133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 923/1838] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:17 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:17.925357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf582a-e86d-4d42-8430-ad47e7d2e0ee admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:17.946643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf582a-e86d-4d42-8430-ad47e7d2e0ee admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:17.952627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d6749-76be-4ec7-817d-bdb4035403b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:17.954475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d6749-76be-4ec7-817d-bdb4035403b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:17.954886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d6749-76be-4ec7-817d-bdb4035403b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:17.961950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d6749-76be-4ec7-817d-bdb4035403b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:17.962797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d6749-76be-4ec7-817d-bdb4035403b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:17.969187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d6749-76be-4ec7-817d-bdb4035403b8 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:17.994758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f98c83-faea-417d-bbaf-b5349b6b5d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:17.996735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f98c83-faea-417d-bbaf-b5349b6b5d82 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:17.997345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f98c83-faea-417d-bbaf-b5349b6b5d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:17.997780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f98c83-faea-417d-bbaf-b5349b6b5d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:18.011939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 924/1839] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:18.013634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d6749-76be-4ec7-817d-bdb4035403b8 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:18.018217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f98c83-faea-417d-bbaf-b5349b6b5d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:18.020007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f98c83-faea-417d-bbaf-b5349b6b5d82 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:18.034441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 916/1840] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:18.050834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d6749-76be-4ec7-817d-bdb4035403b8 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:18.078946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbe380-3d95-43e9-bc3a-dffbcc01ebaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:18.081194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbe380-3d95-43e9-bc3a-dffbcc01ebaf admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:18.081560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbe380-3d95-43e9-bc3a-dffbcc01ebaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:18.082635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbe380-3d95-43e9-bc3a-dffbcc01ebaf admin admin] PATH_INFO: `/v3/projects/11f0a7d143da4eb3bdefd1b805da28f1/users/a06122bd1b3e4eb386f58956e19781a2/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:18.116311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dbe380-3d95-43e9-bc3a-dffbcc01ebaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f0a7d143da4eb3bdefd1b805da28f1, user_id=a06122bd1b3e4eb386f58956e19781a2, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:18.119216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dbe380-3d95-43e9-bc3a-dffbcc01ebaf admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:18.181975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 925/1841] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:18 2022] PUT /identity/v3/projects/11f0a7d143da4eb3bdefd1b805da28f1/users/a06122bd1b3e4eb386f58956e19781a2/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:18.186036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f98c83-faea-417d-bbaf-b5349b6b5d82 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:18.212893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f98c83-faea-417d-bbaf-b5349b6b5d82 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:18.312553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939a5fc-9a55-4dcd-8c0d-a6c89351a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:18.313663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939a5fc-9a55-4dcd-8c0d-a6c89351a444 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:18.314129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939a5fc-9a55-4dcd-8c0d-a6c89351a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:18.314623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939a5fc-9a55-4dcd-8c0d-a6c89351a444 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:18.315707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939a5fc-9a55-4dcd-8c0d-a6c89351a444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:18.318221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939a5fc-9a55-4dcd-8c0d-a6c89351a444 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:18.333003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 917/1842] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:18.337571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbe380-3d95-43e9-bc3a-dffbcc01ebaf admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:18.353837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbe380-3d95-43e9-bc3a-dffbcc01ebaf admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:18.362381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92feccd-2d12-4953-be58-9d4199639c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:18.366399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92feccd-2d12-4953-be58-9d4199639c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:18.366930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92feccd-2d12-4953-be58-9d4199639c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:18.367388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92feccd-2d12-4953-be58-9d4199639c49 admin admin] PATH_INFO: `/v3/projects/11f0a7d143da4eb3bdefd1b805da28f1/users/a06122bd1b3e4eb386f58956e19781a2/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:18.371239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92feccd-2d12-4953-be58-9d4199639c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f0a7d143da4eb3bdefd1b805da28f1, user_id=a06122bd1b3e4eb386f58956e19781a2, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:18.375009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92feccd-2d12-4953-be58-9d4199639c49 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:18.392133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 926/1843] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:18 2022] PUT /identity/v3/projects/11f0a7d143da4eb3bdefd1b805da28f1/users/a06122bd1b3e4eb386f58956e19781a2/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:18.405241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348fea2-5012-4b2f-9752-7b89eaf47e93 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:18.405571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348fea2-5012-4b2f-9752-7b89eaf47e93 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:18.405965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348fea2-5012-4b2f-9752-7b89eaf47e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:18.479135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-7348fea2-5012-4b2f-9752-7b89eaf47e93 None None] MFA Rules not processed for user `a06122bd1b3e4eb386f58956e19781a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:18.481510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7348fea2-5012-4b2f-9752-7b89eaf47e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:18.547199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:18.576646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 918/1844] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:18.587728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92feccd-2d12-4953-be58-9d4199639c49 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:18.590628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a92feccd-2d12-4953-be58-9d4199639c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:18.659909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92feccd-2d12-4953-be58-9d4199639c49 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:18.663793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:18.664463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:18.664781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:18.665065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:18.667501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:18.742643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:18.744222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:18.756495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 927/1845] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:19.396192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348fea2-5012-4b2f-9752-7b89eaf47e93 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:19.397786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:19.399294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7348fea2-5012-4b2f-9752-7b89eaf47e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:19.400715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:19.437730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:19.485294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04efd31a-4bfe-4d0b-99e3-760ec27af578 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:19.491501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:19.492614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:19.493776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:19.493776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] PATH_INFO: `/v3/projects/7e652bdfeb0d42358829e18ac7dcfe03` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:19.494660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e652bdfeb0d42358829e18ac7dcfe03)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:19.497483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:19.522624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348fea2-5012-4b2f-9752-7b89eaf47e93 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:19.532870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:19.536085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:19.536270 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:19.536754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] PATH_INFO: `/v3/users/c18a60a0208b4f88aa8277fea0853d9b` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:19.539775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18a60a0208b4f88aa8277fea0853d9b)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:19.542576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:19.571292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e652bdfeb0d42358829e18ac7dcfe03', 'request_id': 'req-a173b475-752a-4c35-8f83-7107b59ed064'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.583282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e652bdfeb0d42358829e18ac7dcfe03', 'request_id': 'req-a173b475-752a-4c35-8f83-7107b59ed064'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.595044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e652bdfeb0d42358829e18ac7dcfe03', 'request_id': 'req-a173b475-752a-4c35-8f83-7107b59ed064'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.610012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] The token cache is being invalidate because project 7e652bdfeb0d42358829e18ac7dcfe03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:19.610784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.611760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.621132 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 928/1846] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:19 2022] DELETE /identity/v3/projects/7e652bdfeb0d42358829e18ac7dcfe03 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:19.624068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18a60a0208b4f88aa8277fea0853d9b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.624219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:19.625838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:19.638985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18a60a0208b4f88aa8277fea0853d9b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.649050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18a60a0208b4f88aa8277fea0853d9b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.649835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18a60a0208b4f88aa8277fea0853d9b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.658828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-85729f46-2560-476e-b684-a13748dd84a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18a60a0208b4f88aa8277fea0853d9b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.665249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 919/1847] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:19 2022] DELETE /identity/v3/users/c18a60a0208b4f88aa8277fea0853d9b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:19.670530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e99106-b285-4b93-a5c0-4aa96d946ac3 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:19.670749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e99106-b285-4b93-a5c0-4aa96d946ac3 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:19.671159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e99106-b285-4b93-a5c0-4aa96d946ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:19.689773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c0e99106-b285-4b93-a5c0-4aa96d946ac3 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:19.691393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c0e99106-b285-4b93-a5c0-4aa96d946ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:19.706679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173b475-752a-4c35-8f83-7107b59ed064 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:19.767874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 920/1848] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:19.769594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:19.770482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:19.770768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:19.771045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] PATH_INFO: `/v3/users/5d9e35b17c9e49ac86a89afe1ef9ca9a` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:19.772699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9e35b17c9e49ac86a89afe1ef9ca9a)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:19.774615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:19.778475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:19.788150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e99106-b285-4b93-a5c0-4aa96d946ac3 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:19.797375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c0e99106-b285-4b93-a5c0-4aa96d946ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:19.812253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:19.844825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9e35b17c9e49ac86a89afe1ef9ca9a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.866180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9e35b17c9e49ac86a89afe1ef9ca9a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.867255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9e35b17c9e49ac86a89afe1ef9ca9a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.868422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9e35b17c9e49ac86a89afe1ef9ca9a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.881660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e99106-b285-4b93-a5c0-4aa96d946ac3 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:19.883277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-623bdd9d-a4a7-4b0f-a47c-ebe04ee5deb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9e35b17c9e49ac86a89afe1ef9ca9a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:19.891323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:19.892664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:19.892960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:19.893271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:19.898312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:19.903647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 929/1849] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:19 2022] DELETE /identity/v3/users/5d9e35b17c9e49ac86a89afe1ef9ca9a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:19.905834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934e34-7d8c-4070-9e45-eec142d5c0fd None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:19.916614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934e34-7d8c-4070-9e45-eec142d5c0fd None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:19.920794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7934e34-7d8c-4070-9e45-eec142d5c0fd None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:19.926716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 930/1850] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:49:19 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:49:19.929260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:19.935380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dd477-460b-4a24-a2d6-830c31e8e87c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:19.936339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dd477-460b-4a24-a2d6-830c31e8e87c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:19.936339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dd477-460b-4a24-a2d6-830c31e8e87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:19.940398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:19.953526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 921/1851] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:19.975213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-897dd477-460b-4a24-a2d6-830c31e8e87c None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:19.977131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-897dd477-460b-4a24-a2d6-830c31e8e87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:20.111132 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 931/1852] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:49:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:20.129361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:20.129361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:20.249090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d16d7-7997-4ea3-b7ef-5f0f6834ec81 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:20.256297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543ae4d-0ef0-42be-80cd-fbb97c5c4ab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:20.257664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543ae4d-0ef0-42be-80cd-fbb97c5c4ab3 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:20.258031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543ae4d-0ef0-42be-80cd-fbb97c5c4ab3 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:20.258384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1543ae4d-0ef0-42be-80cd-fbb97c5c4ab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:20.269973 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543ae4d-0ef0-42be-80cd-fbb97c5c4ab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:20.272142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1543ae4d-0ef0-42be-80cd-fbb97c5c4ab3 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:20.292796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 922/1853] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:20.527662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897dd477-460b-4a24-a2d6-830c31e8e87c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:20.529443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-897dd477-460b-4a24-a2d6-830c31e8e87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:20.545258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543ae4d-0ef0-42be-80cd-fbb97c5c4ab3 service nova] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:20.547290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1543ae4d-0ef0-42be-80cd-fbb97c5c4ab3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:20.607228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897dd477-460b-4a24-a2d6-830c31e8e87c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:20.612145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:20.613038 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:20.613317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:20.613589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] PATH_INFO: `/v3/projects/807a586c2ca24378b7199be9f51c9136` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:20.614768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=807a586c2ca24378b7199be9f51c9136)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:20.616848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:20.654775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1543ae4d-0ef0-42be-80cd-fbb97c5c4ab3 service nova] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:20.660429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:20.660874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:20.661810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:20.662291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:20.662619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] PATH_INFO: `/v3/projects/73e59ee850944db69fdb9a50588c79fc` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:20.664102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e59ee850944db69fdb9a50588c79fc)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:20.666348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:20.704987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807a586c2ca24378b7199be9f51c9136', 'request_id': 'req-7cae0f56-6e85-4cee-a221-98be18234cd1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.714701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807a586c2ca24378b7199be9f51c9136', 'request_id': 'req-7cae0f56-6e85-4cee-a221-98be18234cd1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.721889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807a586c2ca24378b7199be9f51c9136', 'request_id': 'req-7cae0f56-6e85-4cee-a221-98be18234cd1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.722981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:20.735873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e59ee850944db69fdb9a50588c79fc', 'request_id': 'req-4838ded1-f8b1-441f-b91a-451d3f874e06'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.737280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] The token cache is being invalidate because project 807a586c2ca24378b7199be9f51c9136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:20.737608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.739635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7cae0f56-6e85-4cee-a221-98be18234cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.744677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 932/1854] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:20 2022] DELETE /identity/v3/projects/807a586c2ca24378b7199be9f51c9136 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:20.745434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e59ee850944db69fdb9a50588c79fc', 'request_id': 'req-4838ded1-f8b1-441f-b91a-451d3f874e06'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.746799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867f8a-4a49-4639-bf3f-5fabdb272503 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:20.747572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867f8a-4a49-4639-bf3f-5fabdb272503 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:20.747864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867f8a-4a49-4639-bf3f-5fabdb272503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:20.753580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e59ee850944db69fdb9a50588c79fc', 'request_id': 'req-4838ded1-f8b1-441f-b91a-451d3f874e06'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.765411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] The token cache is being invalidate because project 73e59ee850944db69fdb9a50588c79fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:20.765699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.766509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.768328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 923/1855] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:20 2022] DELETE /identity/v3/projects/73e59ee850944db69fdb9a50588c79fc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:20.771298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a7867f8a-4a49-4639-bf3f-5fabdb272503 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:20.772091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:20.773064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a7867f8a-4a49-4639-bf3f-5fabdb272503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:20.773906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:20.846538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 933/1856] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:20.848891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838ded1-f8b1-441f-b91a-451d3f874e06 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:20.853862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:20.854002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b30e93-2b04-4bf0-b229-87ba7732577e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:20.854854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b30e93-2b04-4bf0-b229-87ba7732577e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:20.855726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:20.855831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b30e93-2b04-4bf0-b229-87ba7732577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:20.856020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:20.856304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] PATH_INFO: `/v3/users/670a63ff653d4b9abc9c5d438e6f01e7` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:20.857552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=670a63ff653d4b9abc9c5d438e6f01e7)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:20.859488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:20.875748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d9b30e93-2b04-4bf0-b229-87ba7732577e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:20.877003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d9b30e93-2b04-4bf0-b229-87ba7732577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:20.885230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 934/1857] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:20.889753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b30e93-2b04-4bf0-b229-87ba7732577e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:20.900549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b30e93-2b04-4bf0-b229-87ba7732577e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:20.906335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144d1f6-b91a-4ad3-ab9d-1901e5caeb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:20.907013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d1f6-b91a-4ad3-ab9d-1901e5caeb90 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:20.907298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d1f6-b91a-4ad3-ab9d-1901e5caeb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:20.907705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d1f6-b91a-4ad3-ab9d-1901e5caeb90 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:20.908436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144d1f6-b91a-4ad3-ab9d-1901e5caeb90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:20.908783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144d1f6-b91a-4ad3-ab9d-1901e5caeb90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:20.909794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '670a63ff653d4b9abc9c5d438e6f01e7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.910461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144d1f6-b91a-4ad3-ab9d-1901e5caeb90 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:20.922813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 935/1858] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:49:20.922956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '670a63ff653d4b9abc9c5d438e6f01e7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.926945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144d1f6-b91a-4ad3-ab9d-1901e5caeb90 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:20.932256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '670a63ff653d4b9abc9c5d438e6f01e7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.933719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '670a63ff653d4b9abc9c5d438e6f01e7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.941397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '670a63ff653d4b9abc9c5d438e6f01e7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:20.942998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144d1f6-b91a-4ad3-ab9d-1901e5caeb90 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:20.947331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 924/1859] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:20 2022] DELETE /identity/v3/users/670a63ff653d4b9abc9c5d438e6f01e7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:20.950274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2caa3-0448-450c-8205-ecc4f545f5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:20.950274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2caa3-0448-450c-8205-ecc4f545f5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:20.950274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2caa3-0448-450c-8205-ecc4f545f5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:20.950274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2caa3-0448-450c-8205-ecc4f545f5f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:20.950649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2caa3-0448-450c-8205-ecc4f545f5f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:20.951822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a2caa3-0448-450c-8205-ecc4f545f5f1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:20.980637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 936/1860] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:20 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:49:20.985146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:20.986546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:21.050515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:21.074039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de91a04-793f-416a-a759-391daf5b49bc admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.143874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2948de-12bd-4961-b496-8450339e1501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.144618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2948de-12bd-4961-b496-8450339e1501 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.144883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2948de-12bd-4961-b496-8450339e1501 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.145164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2948de-12bd-4961-b496-8450339e1501 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.145869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2948de-12bd-4961-b496-8450339e1501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.148320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2948de-12bd-4961-b496-8450339e1501 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.179454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 925/1861] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:20 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:21.180078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2caa3-0448-450c-8205-ecc4f545f5f1 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.197104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2caa3-0448-450c-8205-ecc4f545f5f1 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.202806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fd56d-e331-4500-8b38-fca29a43a32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.204203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fd56d-e331-4500-8b38-fca29a43a32a admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.204494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fd56d-e331-4500-8b38-fca29a43a32a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.204810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fd56d-e331-4500-8b38-fca29a43a32a admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.206079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fd56d-e331-4500-8b38-fca29a43a32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.208029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fd56d-e331-4500-8b38-fca29a43a32a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.225032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 937/1862] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:21.227074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2948de-12bd-4961-b496-8450339e1501 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.237473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2948de-12bd-4961-b496-8450339e1501 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.242456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b9960-4e19-4084-9169-ee018081b2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.243289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b9960-4e19-4084-9169-ee018081b2fc admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.243577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b9960-4e19-4084-9169-ee018081b2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.243856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b9960-4e19-4084-9169-ee018081b2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.244618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b9960-4e19-4084-9169-ee018081b2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.246383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b9960-4e19-4084-9169-ee018081b2fc admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.265265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 926/1863] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:21.266517 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fd56d-e331-4500-8b38-fca29a43a32a admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.278372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fd56d-e331-4500-8b38-fca29a43a32a admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.283131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b5db0-fa2f-41e8-b3e4-4a26a895e3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.284040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b5db0-fa2f-41e8-b3e4-4a26a895e3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.284328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b5db0-fa2f-41e8-b3e4-4a26a895e3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.284645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b5db0-fa2f-41e8-b3e4-4a26a895e3eb admin admin] PATH_INFO: `/v3/projects/1f00ee7538da46e388299f14deeeac82/users/aecf96229a2440fd989f5ebd5a79a9af/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.301016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b5db0-fa2f-41e8-b3e4-4a26a895e3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f00ee7538da46e388299f14deeeac82, user_id=aecf96229a2440fd989f5ebd5a79a9af, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.303805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b5db0-fa2f-41e8-b3e4-4a26a895e3eb admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.317857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 938/1864] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:21 2022] PUT /identity/v3/projects/1f00ee7538da46e388299f14deeeac82/users/aecf96229a2440fd989f5ebd5a79a9af/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:21.328609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9862082-b88a-43fa-8ca4-6bfa495baa7a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.328983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9862082-b88a-43fa-8ca4-6bfa495baa7a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.329360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9862082-b88a-43fa-8ca4-6bfa495baa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.358814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a9862082-b88a-43fa-8ca4-6bfa495baa7a None None] MFA Rules not processed for user `aecf96229a2440fd989f5ebd5a79a9af`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:21.360149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a9862082-b88a-43fa-8ca4-6bfa495baa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:21.400891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b5db0-fa2f-41e8-b3e4-4a26a895e3eb admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.403255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ce8b5db0-fa2f-41e8-b3e4-4a26a895e3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:21.443375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:21.447351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 927/1865] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:21.451870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9862082-b88a-43fa-8ca4-6bfa495baa7a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.485620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b5db0-fa2f-41e8-b3e4-4a26a895e3eb admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.490645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.491512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.491797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.492078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] PATH_INFO: `/v3/projects/4baa775b0dbe401e94bdafc44c4fcb55` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.493293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4baa775b0dbe401e94bdafc44c4fcb55)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.495383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.524981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9862082-b88a-43fa-8ca4-6bfa495baa7a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.535057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368bde5-e2ee-4821-82ec-9ff4ae110fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.535773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368bde5-e2ee-4821-82ec-9ff4ae110fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.536128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368bde5-e2ee-4821-82ec-9ff4ae110fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.536469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368bde5-e2ee-4821-82ec-9ff4ae110fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.537268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368bde5-e2ee-4821-82ec-9ff4ae110fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.539162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c368bde5-e2ee-4821-82ec-9ff4ae110fa0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.542655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:21.547824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4baa775b0dbe401e94bdafc44c4fcb55', 'request_id': 'req-f182e08f-515a-4997-870e-571bc92a5807'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.556219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4baa775b0dbe401e94bdafc44c4fcb55', 'request_id': 'req-f182e08f-515a-4997-870e-571bc92a5807'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.567769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4baa775b0dbe401e94bdafc44c4fcb55', 'request_id': 'req-f182e08f-515a-4997-870e-571bc92a5807'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.580735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 928/1866] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:21 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:49:21.584536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368bde5-e2ee-4821-82ec-9ff4ae110fa0 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.588040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] The token cache is being invalidate because project 4baa775b0dbe401e94bdafc44c4fcb55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:21.588334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.589082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.593463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 939/1867] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:21 2022] DELETE /identity/v3/projects/4baa775b0dbe401e94bdafc44c4fcb55 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:21.599069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.601261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:21.659206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368bde5-e2ee-4821-82ec-9ff4ae110fa0 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.665204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c649fea-d200-4932-9d03-038c4e02aa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.666087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c649fea-d200-4932-9d03-038c4e02aa16 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.666816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c649fea-d200-4932-9d03-038c4e02aa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.667866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c649fea-d200-4932-9d03-038c4e02aa16 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.668951 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c649fea-d200-4932-9d03-038c4e02aa16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.671250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c649fea-d200-4932-9d03-038c4e02aa16 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.687145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182e08f-515a-4997-870e-571bc92a5807 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.694400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.695740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.696203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.697349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] PATH_INFO: `/v3/users/1889b65a873e41528054e735df8f7e86` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.699923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1889b65a873e41528054e735df8f7e86)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.702753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.703758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 929/1868] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:21 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:49:21.707809 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c649fea-d200-4932-9d03-038c4e02aa16 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.710918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0c649fea-d200-4932-9d03-038c4e02aa16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:21.726353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c649fea-d200-4932-9d03-038c4e02aa16 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.735154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acb7c1-0ab1-4a7c-8677-3afc624c66bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.737025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acb7c1-0ab1-4a7c-8677-3afc624c66bc admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.737381 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acb7c1-0ab1-4a7c-8677-3afc624c66bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.737898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acb7c1-0ab1-4a7c-8677-3afc624c66bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.738926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acb7c1-0ab1-4a7c-8677-3afc624c66bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.741528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acb7c1-0ab1-4a7c-8677-3afc624c66bc admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.757356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 930/1869] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:49:21.762800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acb7c1-0ab1-4a7c-8677-3afc624c66bc admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.780660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acb7c1-0ab1-4a7c-8677-3afc624c66bc admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.782012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1889b65a873e41528054e735df8f7e86'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.785840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc647fc-dfdd-422b-a1ba-53c42fadb97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:21.787137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc647fc-dfdd-422b-a1ba-53c42fadb97a admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:21.787568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc647fc-dfdd-422b-a1ba-53c42fadb97a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:21.787995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc647fc-dfdd-422b-a1ba-53c42fadb97a admin admin] PATH_INFO: `/v3/projects/deab58d16de0482cacc0eaaa70a6af5b/users/8f1f22fefc7d4feba506d38de2f18f95/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:21.801648 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:21.814440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1889b65a873e41528054e735df8f7e86'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.816271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1889b65a873e41528054e735df8f7e86'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.817280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1889b65a873e41528054e735df8f7e86'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.817868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc647fc-dfdd-422b-a1ba-53c42fadb97a admin admin] RBAC: Authorizing `identity:create_grant(project_id=deab58d16de0482cacc0eaaa70a6af5b, user_id=8f1f22fefc7d4feba506d38de2f18f95, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:21.822505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc647fc-dfdd-422b-a1ba-53c42fadb97a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:21.832211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1889b65a873e41528054e735df8f7e86'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:21.839427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 931/1870] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:21 2022] PUT /identity/v3/projects/deab58d16de0482cacc0eaaa70a6af5b/users/8f1f22fefc7d4feba506d38de2f18f95/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:21.846771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc647fc-dfdd-422b-a1ba-53c42fadb97a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:21.855163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5cc647fc-dfdd-422b-a1ba-53c42fadb97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:21.864249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 940/1871] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:21 2022] DELETE /identity/v3/users/1889b65a873e41528054e735df8f7e86 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:21.938007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc647fc-dfdd-422b-a1ba-53c42fadb97a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:21.990462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:22.006374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8726f-56eb-4035-a9de-739185206f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:22.007188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8726f-56eb-4035-a9de-739185206f8d admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:22.007461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8726f-56eb-4035-a9de-739185206f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:22.007743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8726f-56eb-4035-a9de-739185206f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:22.008483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8726f-56eb-4035-a9de-739185206f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:22.010120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8726f-56eb-4035-a9de-739185206f8d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:22.024631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 932/1872] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:22.025934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:22.036789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26fd54-4047-4c8a-8cb2-fd8a85ffe925 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:22.042192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e4ce5-bb31-4a12-9c82-eaa650bc5857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:22.043117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e4ce5-bb31-4a12-9c82-eaa650bc5857 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:22.043419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e4ce5-bb31-4a12-9c82-eaa650bc5857 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:22.043697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e4ce5-bb31-4a12-9c82-eaa650bc5857 admin admin] PATH_INFO: `/v3/projects/deab58d16de0482cacc0eaaa70a6af5b/users/8f1f22fefc7d4feba506d38de2f18f95/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:22.045788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e4ce5-bb31-4a12-9c82-eaa650bc5857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deab58d16de0482cacc0eaaa70a6af5b, user_id=8f1f22fefc7d4feba506d38de2f18f95, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:22.048481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e4ce5-bb31-4a12-9c82-eaa650bc5857 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:22.060522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 941/1873] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:22 2022] PUT /identity/v3/projects/deab58d16de0482cacc0eaaa70a6af5b/users/8f1f22fefc7d4feba506d38de2f18f95/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:22.071997 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e41ffc-bca2-41ed-9b17-7226fdbe7ca3 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:22.072294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e41ffc-bca2-41ed-9b17-7226fdbe7ca3 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:22.072613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e41ffc-bca2-41ed-9b17-7226fdbe7ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:22.102242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c8e41ffc-bca2-41ed-9b17-7226fdbe7ca3 None None] MFA Rules not processed for user `8f1f22fefc7d4feba506d38de2f18f95`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:22.103478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c8e41ffc-bca2-41ed-9b17-7226fdbe7ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:22.175576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 933/1874] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:22.186703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e4ce5-bb31-4a12-9c82-eaa650bc5857 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:22.188426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-462e4ce5-bb31-4a12-9c82-eaa650bc5857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:22.253230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e4ce5-bb31-4a12-9c82-eaa650bc5857 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:22.265537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f56911-f377-4ad3-b35f-dff8dfc130aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:22.266231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f56911-f377-4ad3-b35f-dff8dfc130aa service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:22.266565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f56911-f377-4ad3-b35f-dff8dfc130aa service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:22.266849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f56911-f377-4ad3-b35f-dff8dfc130aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:22.278102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f56911-f377-4ad3-b35f-dff8dfc130aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:22.279558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f56911-f377-4ad3-b35f-dff8dfc130aa service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:22.290974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 942/1875] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:22.456242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e41ffc-bca2-41ed-9b17-7226fdbe7ca3 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:22.458549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c8e41ffc-bca2-41ed-9b17-7226fdbe7ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:22.596675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e41ffc-bca2-41ed-9b17-7226fdbe7ca3 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:22.602354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:22.603805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:22.603805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:22.604703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] PATH_INFO: `/v3/projects/77d38b902ca044b5ba0ee012128da294` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:22.606394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d38b902ca044b5ba0ee012128da294)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:22.608350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:22.696103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d38b902ca044b5ba0ee012128da294', 'request_id': 'req-cfab96fe-191c-4158-8e48-23a3b0adfd14'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:22.705389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d38b902ca044b5ba0ee012128da294', 'request_id': 'req-cfab96fe-191c-4158-8e48-23a3b0adfd14'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:22.716645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d38b902ca044b5ba0ee012128da294', 'request_id': 'req-cfab96fe-191c-4158-8e48-23a3b0adfd14'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:22.724883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:22.732554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] The token cache is being invalidate because project 77d38b902ca044b5ba0ee012128da294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:22.732914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:22.737054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-cfab96fe-191c-4158-8e48-23a3b0adfd14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:22.740299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 934/1876] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:22 2022] DELETE /identity/v3/projects/77d38b902ca044b5ba0ee012128da294 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:22.746172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d450dfd-e73b-4712-853b-3cb8142f1ded None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:22.746625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d450dfd-e73b-4712-853b-3cb8142f1ded None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:22.746863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d450dfd-e73b-4712-853b-3cb8142f1ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:22.765580 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-5d450dfd-e73b-4712-853b-3cb8142f1ded None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:22.766814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5d450dfd-e73b-4712-853b-3cb8142f1ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:22.837221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 943/1877] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:22.846469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4464e79-95ad-4d57-bd9b-1acaa47d0073 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:22.846917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4464e79-95ad-4d57-bd9b-1acaa47d0073 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:22.847262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4464e79-95ad-4d57-bd9b-1acaa47d0073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:22.866993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f4464e79-95ad-4d57-bd9b-1acaa47d0073 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:22.868209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f4464e79-95ad-4d57-bd9b-1acaa47d0073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:22.880619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 935/1878] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:22.883127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d450dfd-e73b-4712-853b-3cb8142f1ded None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:22.894484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d450dfd-e73b-4712-853b-3cb8142f1ded None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:22.900043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068df3a-23df-4d0b-8e06-42c71cdaa061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:22.900758 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068df3a-23df-4d0b-8e06-42c71cdaa061 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:22.901673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068df3a-23df-4d0b-8e06-42c71cdaa061 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:22.902137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068df3a-23df-4d0b-8e06-42c71cdaa061 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:22.903094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068df3a-23df-4d0b-8e06-42c71cdaa061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:22.903434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068df3a-23df-4d0b-8e06-42c71cdaa061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:22.905194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068df3a-23df-4d0b-8e06-42c71cdaa061 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:22.922302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 944/1879] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:49:22.923451 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4464e79-95ad-4d57-bd9b-1acaa47d0073 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:22.934246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4464e79-95ad-4d57-bd9b-1acaa47d0073 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:22.940033 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd90d7-938e-46cb-82c0-b360878ab509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:22.941019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd90d7-938e-46cb-82c0-b360878ab509 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:22.941301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd90d7-938e-46cb-82c0-b360878ab509 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:22.941652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd90d7-938e-46cb-82c0-b360878ab509 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:22.942362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd90d7-938e-46cb-82c0-b360878ab509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:22.944523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd90d7-938e-46cb-82c0-b360878ab509 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:22.988052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 936/1880] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:22 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:22.989617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068df3a-23df-4d0b-8e06-42c71cdaa061 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:23.004892 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068df3a-23df-4d0b-8e06-42c71cdaa061 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:23.075666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd90d7-938e-46cb-82c0-b360878ab509 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:23.077515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-eecd90d7-938e-46cb-82c0-b360878ab509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:23.088210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:23.101706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a058229-77d0-489d-81a7-c4ce183d8567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:23.102510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a058229-77d0-489d-81a7-c4ce183d8567 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.103015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a058229-77d0-489d-81a7-c4ce183d8567 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.103321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a058229-77d0-489d-81a7-c4ce183d8567 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.105514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a058229-77d0-489d-81a7-c4ce183d8567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:23.107637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a058229-77d0-489d-81a7-c4ce183d8567 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:23.146394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 945/1881] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:22 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:49:23.153604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a058229-77d0-489d-81a7-c4ce183d8567 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:23.169096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd90d7-938e-46cb-82c0-b360878ab509 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:23.169391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a058229-77d0-489d-81a7-c4ce183d8567 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:23.173842 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:23.173996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f317d78-f8d4-477a-b006-d3925e280ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:23.174637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f317d78-f8d4-477a-b006-d3925e280ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.174759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.174932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f317d78-f8d4-477a-b006-d3925e280ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.175050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.175215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f317d78-f8d4-477a-b006-d3925e280ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.175306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] PATH_INFO: `/v3/users/aecf96229a2440fd989f5ebd5a79a9af` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.175954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f317d78-f8d4-477a-b006-d3925e280ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:23.176485 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] RBAC: Authorizing `identity:delete_user(user_id=aecf96229a2440fd989f5ebd5a79a9af)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:23.177598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f317d78-f8d4-477a-b006-d3925e280ee3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:23.178410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:23.188768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 946/1882] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:23.192560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f317d78-f8d4-477a-b006-d3925e280ee3 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:23.203085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f317d78-f8d4-477a-b006-d3925e280ee3 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:23.208085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd1205-0c52-495f-b15e-a5428750dc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:23.208825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd1205-0c52-495f-b15e-a5428750dc74 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.209104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd1205-0c52-495f-b15e-a5428750dc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.209377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd1205-0c52-495f-b15e-a5428750dc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.210149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd1205-0c52-495f-b15e-a5428750dc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:23.211678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd1205-0c52-495f-b15e-a5428750dc74 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:23.223817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 947/1883] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:23.227884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd1205-0c52-495f-b15e-a5428750dc74 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:23.230364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aecf96229a2440fd989f5ebd5a79a9af'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:23.237945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd1205-0c52-495f-b15e-a5428750dc74 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:23.242481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8dbda-1411-47e7-9a7f-9ffc8ff277e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:23.242767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aecf96229a2440fd989f5ebd5a79a9af'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:23.243338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8dbda-1411-47e7-9a7f-9ffc8ff277e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.243624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8dbda-1411-47e7-9a7f-9ffc8ff277e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.243899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8dbda-1411-47e7-9a7f-9ffc8ff277e8 admin admin] PATH_INFO: `/v3/projects/d1df784ba9ff44689227952c58189999/users/d88f214d857c4b218f73207e8309a03c/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.251016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aecf96229a2440fd989f5ebd5a79a9af'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:23.251710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aecf96229a2440fd989f5ebd5a79a9af'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:23.258802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-b08bc080-03d1-47f1-9134-02cfbb5350df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aecf96229a2440fd989f5ebd5a79a9af'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:23.261118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8dbda-1411-47e7-9a7f-9ffc8ff277e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1df784ba9ff44689227952c58189999, user_id=d88f214d857c4b218f73207e8309a03c, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:23.265612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8dbda-1411-47e7-9a7f-9ffc8ff277e8 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:23.265767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 937/1884] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:23 2022] DELETE /identity/v3/users/aecf96229a2440fd989f5ebd5a79a9af => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:23.269780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878daaf-e884-498b-8ace-d099c05baf74 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.270225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878daaf-e884-498b-8ace-d099c05baf74 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.270419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878daaf-e884-498b-8ace-d099c05baf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.282771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 948/1885] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:23 2022] PUT /identity/v3/projects/d1df784ba9ff44689227952c58189999/users/d88f214d857c4b218f73207e8309a03c/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:23.290152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-1878daaf-e884-498b-8ace-d099c05baf74 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:23.291371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1878daaf-e884-498b-8ace-d099c05baf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:23.294577 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e05a82-321e-49f9-b777-eb2f929b9855 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.294967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e05a82-321e-49f9-b777-eb2f929b9855 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.296268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e05a82-321e-49f9-b777-eb2f929b9855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.327423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-70e05a82-321e-49f9-b777-eb2f929b9855 None None] MFA Rules not processed for user `d88f214d857c4b218f73207e8309a03c`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:23.328597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-70e05a82-321e-49f9-b777-eb2f929b9855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:23.339176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:23.363699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 938/1886] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:23.373128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878daaf-e884-498b-8ace-d099c05baf74 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:23.374498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1878daaf-e884-498b-8ace-d099c05baf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:23.406486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 949/1887] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:23.418842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e05a82-321e-49f9-b777-eb2f929b9855 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:23.420840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-70e05a82-321e-49f9-b777-eb2f929b9855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:23.443328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878daaf-e884-498b-8ace-d099c05baf74 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:23.447403 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32854-5514-4f39-a750-f96282b3cfb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:23.448720 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32854-5514-4f39-a750-f96282b3cfb5 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.449167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32854-5514-4f39-a750-f96282b3cfb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.449543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32854-5514-4f39-a750-f96282b3cfb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.459828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a32854-5514-4f39-a750-f96282b3cfb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:23.461416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a32854-5514-4f39-a750-f96282b3cfb5 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:23.471599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 939/1888] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:49:23.489074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e05a82-321e-49f9-b777-eb2f929b9855 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:23.493001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f46b-b606-4e94-a71a-1433946d352b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:23.493773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f46b-b606-4e94-a71a-1433946d352b service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.494213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f46b-b606-4e94-a71a-1433946d352b service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.494510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f46b-b606-4e94-a71a-1433946d352b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.503647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84f46b-b606-4e94-a71a-1433946d352b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:23.505158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84f46b-b606-4e94-a71a-1433946d352b service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:23.514880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 950/1889] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:49:23.641823 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32854-5514-4f39-a750-f96282b3cfb5 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:23.643280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-99a32854-5514-4f39-a750-f96282b3cfb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:23.664065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32854-5514-4f39-a750-f96282b3cfb5 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:23.673171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a74be-06fd-4635-ac4d-8af069aba808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:23.673887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a74be-06fd-4635-ac4d-8af069aba808 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:23.674355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a74be-06fd-4635-ac4d-8af069aba808 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:23.675252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a74be-06fd-4635-ac4d-8af069aba808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:23.679095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f17a74be-06fd-4635-ac4d-8af069aba808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:23.809634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a74be-06fd-4635-ac4d-8af069aba808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:23.811490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a74be-06fd-4635-ac4d-8af069aba808 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:23.823414 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 940/1890] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:23.975965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f46b-b606-4e94-a71a-1433946d352b service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:23.978080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-aa84f46b-b606-4e94-a71a-1433946d352b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:24.005712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f46b-b606-4e94-a71a-1433946d352b service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:24.029371 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:24.030599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:24.030829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:24.031104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] PATH_INFO: `/v3/projects/1f00ee7538da46e388299f14deeeac82` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:24.038263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f00ee7538da46e388299f14deeeac82)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:24.041057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:24.098611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:24.128910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f00ee7538da46e388299f14deeeac82', 'request_id': 'req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.142710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f00ee7538da46e388299f14deeeac82', 'request_id': 'req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.152361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f00ee7538da46e388299f14deeeac82', 'request_id': 'req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.170996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] The token cache is being invalidate because project 1f00ee7538da46e388299f14deeeac82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:24.171908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.174355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.176961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 951/1891] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:23 2022] DELETE /identity/v3/projects/1f00ee7538da46e388299f14deeeac82 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:24.180795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a74be-06fd-4635-ac4d-8af069aba808 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:24.193092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a74be-06fd-4635-ac4d-8af069aba808 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:24.287889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:24.288803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:24.289087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:24.289357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] PATH_INFO: `/v3/users/8f1f22fefc7d4feba506d38de2f18f95` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:24.290542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f1f22fefc7d4feba506d38de2f18f95)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:24.293261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:24.324694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:24.354194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f1f22fefc7d4feba506d38de2f18f95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.381515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f1f22fefc7d4feba506d38de2f18f95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.390146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f1f22fefc7d4feba506d38de2f18f95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.390764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f1f22fefc7d4feba506d38de2f18f95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.398198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-a6361f12-5ac6-4816-92c0-a0316c2f724f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f1f22fefc7d4feba506d38de2f18f95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.405362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 941/1892] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:24 2022] DELETE /identity/v3/users/8f1f22fefc7d4feba506d38de2f18f95 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:24.681185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:24.683201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:24.706464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5f0ec-405f-4fc4-b41a-e4c373c22a84 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:24.760958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:24.761818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:24.762105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:24.762377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] PATH_INFO: `/v3/projects/deab58d16de0482cacc0eaaa70a6af5b` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:24.763541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deab58d16de0482cacc0eaaa70a6af5b)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:24.765465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:24.810378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deab58d16de0482cacc0eaaa70a6af5b', 'request_id': 'req-10cf1c7d-86b9-4bb9-a85c-939c40397b33'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.817068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deab58d16de0482cacc0eaaa70a6af5b', 'request_id': 'req-10cf1c7d-86b9-4bb9-a85c-939c40397b33'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.823442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deab58d16de0482cacc0eaaa70a6af5b', 'request_id': 'req-10cf1c7d-86b9-4bb9-a85c-939c40397b33'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.834705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] The token cache is being invalidate because project deab58d16de0482cacc0eaaa70a6af5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:24.834994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.835637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-10cf1c7d-86b9-4bb9-a85c-939c40397b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:24.837420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 952/1893] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:24 2022] DELETE /identity/v3/projects/deab58d16de0482cacc0eaaa70a6af5b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:24.842693 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985626a-c611-4922-9765-3ce7476add6a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:24.843175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985626a-c611-4922-9765-3ce7476add6a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:24.843490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985626a-c611-4922-9765-3ce7476add6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:24.861858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-5985626a-c611-4922-9765-3ce7476add6a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:24.863048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5985626a-c611-4922-9765-3ce7476add6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:24.925303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 942/1894] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:24.933314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f755596-fd35-4073-a127-9bca50eab1cb None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:24.933606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f755596-fd35-4073-a127-9bca50eab1cb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:24.933892 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f755596-fd35-4073-a127-9bca50eab1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:24.952553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0f755596-fd35-4073-a127-9bca50eab1cb None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:24.953783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0f755596-fd35-4073-a127-9bca50eab1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:24.965378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 953/1895] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:24.969784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985626a-c611-4922-9765-3ce7476add6a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:24.980466 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985626a-c611-4922-9765-3ce7476add6a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:24.985525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a364d-bf2d-4299-9bcf-cc305ae52e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:24.986255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a364d-bf2d-4299-9bcf-cc305ae52e59 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:24.986586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a364d-bf2d-4299-9bcf-cc305ae52e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:24.986938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4a364d-bf2d-4299-9bcf-cc305ae52e59 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:24.987906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a364d-bf2d-4299-9bcf-cc305ae52e59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:24.988218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a364d-bf2d-4299-9bcf-cc305ae52e59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:24.989834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4a364d-bf2d-4299-9bcf-cc305ae52e59 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:25.007193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 943/1896] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:49:25.011793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f755596-fd35-4073-a127-9bca50eab1cb None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:25.021793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f755596-fd35-4073-a127-9bca50eab1cb None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:25.026749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898e96c-f36f-41c0-a0dd-32438ee59701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:25.027418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898e96c-f36f-41c0-a0dd-32438ee59701 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:25.027700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898e96c-f36f-41c0-a0dd-32438ee59701 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:25.027979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898e96c-f36f-41c0-a0dd-32438ee59701 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:25.028713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898e96c-f36f-41c0-a0dd-32438ee59701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:25.030615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898e96c-f36f-41c0-a0dd-32438ee59701 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:25.060802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 954/1897] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:25 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:25.064965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a364d-bf2d-4299-9bcf-cc305ae52e59 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:25.082830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4a364d-bf2d-4299-9bcf-cc305ae52e59 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:25.149023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec825c26-f9d6-46e7-b59c-12fa68dab106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:25.149887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec825c26-f9d6-46e7-b59c-12fa68dab106 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:25.150284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec825c26-f9d6-46e7-b59c-12fa68dab106 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:25.150568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec825c26-f9d6-46e7-b59c-12fa68dab106 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:25.151308 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec825c26-f9d6-46e7-b59c-12fa68dab106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:25.153263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec825c26-f9d6-46e7-b59c-12fa68dab106 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:25.180048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 944/1898] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:25 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:25.184432 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898e96c-f36f-41c0-a0dd-32438ee59701 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:25.195026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898e96c-f36f-41c0-a0dd-32438ee59701 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:25.200374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da530521-4f5d-4980-8b55-97d9455bf443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:25.201092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da530521-4f5d-4980-8b55-97d9455bf443 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:25.201375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da530521-4f5d-4980-8b55-97d9455bf443 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:25.201772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da530521-4f5d-4980-8b55-97d9455bf443 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:25.202511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da530521-4f5d-4980-8b55-97d9455bf443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:25.204116 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da530521-4f5d-4980-8b55-97d9455bf443 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:25.217041 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 955/1899] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:25.221185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec825c26-f9d6-46e7-b59c-12fa68dab106 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:25.231888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec825c26-f9d6-46e7-b59c-12fa68dab106 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:25.236655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654ed90-a4bb-462f-be24-ec0a8b2cebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:25.237332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654ed90-a4bb-462f-be24-ec0a8b2cebb9 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:25.237617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654ed90-a4bb-462f-be24-ec0a8b2cebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:25.237897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654ed90-a4bb-462f-be24-ec0a8b2cebb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:25.238627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654ed90-a4bb-462f-be24-ec0a8b2cebb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:25.240280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654ed90-a4bb-462f-be24-ec0a8b2cebb9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:25.251210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 945/1900] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:25.255475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da530521-4f5d-4980-8b55-97d9455bf443 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:25.261880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:25.273186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da530521-4f5d-4980-8b55-97d9455bf443 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:25.281611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b2321-e323-4f0e-8d97-1b35d63277b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:25.282613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b2321-e323-4f0e-8d97-1b35d63277b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:25.282959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b2321-e323-4f0e-8d97-1b35d63277b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:25.283302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b2321-e323-4f0e-8d97-1b35d63277b2 admin admin] PATH_INFO: `/v3/projects/56c4740ad6074a03b54fa9c6bc93ffd8/users/dedcdb7aedf642e8b19addcb8c268cb8/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:25.301144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b2321-e323-4f0e-8d97-1b35d63277b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c4740ad6074a03b54fa9c6bc93ffd8, user_id=dedcdb7aedf642e8b19addcb8c268cb8, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:25.303858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b2321-e323-4f0e-8d97-1b35d63277b2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:25.318352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 956/1901] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:25 2022] PUT /identity/v3/projects/56c4740ad6074a03b54fa9c6bc93ffd8/users/dedcdb7aedf642e8b19addcb8c268cb8/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:25.324191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae6549-150d-457d-b32e-cf33cffd7dc1 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:25.324616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae6549-150d-457d-b32e-cf33cffd7dc1 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:25.324971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae6549-150d-457d-b32e-cf33cffd7dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:25.343157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-79ae6549-150d-457d-b32e-cf33cffd7dc1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:25.344475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-79ae6549-150d-457d-b32e-cf33cffd7dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:25.350008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:25.416436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 946/1902] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:25.429565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b2321-e323-4f0e-8d97-1b35d63277b2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:25.431400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c19b2321-e323-4f0e-8d97-1b35d63277b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:25.505030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b2321-e323-4f0e-8d97-1b35d63277b2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:25.508802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cd32b-d7c8-4aec-9945-ced8af722d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:25.509453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cd32b-d7c8-4aec-9945-ced8af722d95 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:25.509738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cd32b-d7c8-4aec-9945-ced8af722d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:25.510016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cd32b-d7c8-4aec-9945-ced8af722d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:25.520179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cd32b-d7c8-4aec-9945-ced8af722d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:25.521894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818cd32b-d7c8-4aec-9945-ced8af722d95 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:25.535337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 957/1903] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:49:26.204851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d419d1b-b610-409a-a252-7cd84afd7f1a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:26.205890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d419d1b-b610-409a-a252-7cd84afd7f1a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:26.206708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d419d1b-b610-409a-a252-7cd84afd7f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:26.236294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-3d419d1b-b610-409a-a252-7cd84afd7f1a None None] MFA Rules not processed for user `dedcdb7aedf642e8b19addcb8c268cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:26.237836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3d419d1b-b610-409a-a252-7cd84afd7f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:26.332475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 947/1904] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:26.341417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cd32b-d7c8-4aec-9945-ced8af722d95 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:26.351423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cd32b-d7c8-4aec-9945-ced8af722d95 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:26.356955 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f255c56-7cd5-44ed-87a7-2dcc1dc383bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:26.357700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f255c56-7cd5-44ed-87a7-2dcc1dc383bf admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:26.358209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f255c56-7cd5-44ed-87a7-2dcc1dc383bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:26.358671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f255c56-7cd5-44ed-87a7-2dcc1dc383bf admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:26.359435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f255c56-7cd5-44ed-87a7-2dcc1dc383bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:26.361531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f255c56-7cd5-44ed-87a7-2dcc1dc383bf admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:26.399302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 958/1905] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:26 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:26.400518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d419d1b-b610-409a-a252-7cd84afd7f1a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:26.418866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d419d1b-b610-409a-a252-7cd84afd7f1a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:26.490730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20615d65-ed4b-4fa5-a53f-d579967ea08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:26.491734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20615d65-ed4b-4fa5-a53f-d579967ea08b admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:26.491734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20615d65-ed4b-4fa5-a53f-d579967ea08b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:26.492371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20615d65-ed4b-4fa5-a53f-d579967ea08b admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:26.492696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20615d65-ed4b-4fa5-a53f-d579967ea08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:26.495935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20615d65-ed4b-4fa5-a53f-d579967ea08b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:26.516612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 948/1906] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:26 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:26.520929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f255c56-7cd5-44ed-87a7-2dcc1dc383bf admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:26.532828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f255c56-7cd5-44ed-87a7-2dcc1dc383bf admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:26.542810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13149632-cc66-4cc3-8f01-4a303cbdc5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:26.543593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13149632-cc66-4cc3-8f01-4a303cbdc5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:26.543882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13149632-cc66-4cc3-8f01-4a303cbdc5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:26.544273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13149632-cc66-4cc3-8f01-4a303cbdc5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:26.545493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13149632-cc66-4cc3-8f01-4a303cbdc5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:26.547934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13149632-cc66-4cc3-8f01-4a303cbdc5f3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:26.561442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 959/1907] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:49:26.565750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20615d65-ed4b-4fa5-a53f-d579967ea08b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:26.576399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20615d65-ed4b-4fa5-a53f-d579967ea08b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:26.581514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0dfca-9657-477f-b523-52aef9992cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:26.582460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0dfca-9657-477f-b523-52aef9992cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:26.582843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0dfca-9657-477f-b523-52aef9992cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:26.583128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0dfca-9657-477f-b523-52aef9992cbe admin admin] PATH_INFO: `/v3/projects/345f1057c6ad4abe90b005c974378e82/users/aac85b79585049608c74d181add6ed35/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:26.602073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0dfca-9657-477f-b523-52aef9992cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=345f1057c6ad4abe90b005c974378e82, user_id=aac85b79585049608c74d181add6ed35, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:26.605462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0dfca-9657-477f-b523-52aef9992cbe admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:26.630086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 949/1908] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:26 2022] PUT /identity/v3/projects/345f1057c6ad4abe90b005c974378e82/users/aac85b79585049608c74d181add6ed35/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:26.633996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13149632-cc66-4cc3-8f01-4a303cbdc5f3 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:26.651255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13149632-cc66-4cc3-8f01-4a303cbdc5f3 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:26.717091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38004880-19ea-4908-88c1-01db78e5dd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:26.717773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38004880-19ea-4908-88c1-01db78e5dd24 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:26.718057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38004880-19ea-4908-88c1-01db78e5dd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:26.718336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38004880-19ea-4908-88c1-01db78e5dd24 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:26.719069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38004880-19ea-4908-88c1-01db78e5dd24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:26.720826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38004880-19ea-4908-88c1-01db78e5dd24 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:26.731597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 960/1909] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:26.735922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0dfca-9657-477f-b523-52aef9992cbe admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:26.747614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0dfca-9657-477f-b523-52aef9992cbe admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:26.752663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d800-a6e2-43ee-9547-4de344ee2302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:26.753554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d800-a6e2-43ee-9547-4de344ee2302 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:26.753833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d800-a6e2-43ee-9547-4de344ee2302 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:26.754108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d800-a6e2-43ee-9547-4de344ee2302 admin admin] PATH_INFO: `/v3/projects/345f1057c6ad4abe90b005c974378e82/users/aac85b79585049608c74d181add6ed35/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:26.756226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764d800-a6e2-43ee-9547-4de344ee2302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345f1057c6ad4abe90b005c974378e82, user_id=aac85b79585049608c74d181add6ed35, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:26.758883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764d800-a6e2-43ee-9547-4de344ee2302 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:26.761653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:26.771866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 950/1910] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:26 2022] PUT /identity/v3/projects/345f1057c6ad4abe90b005c974378e82/users/aac85b79585049608c74d181add6ed35/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:27.408041 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c78d3-cf85-4546-b200-d4a4012c1901 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:27.409145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c78d3-cf85-4546-b200-d4a4012c1901 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:27.409504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c78d3-cf85-4546-b200-d4a4012c1901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:27.439666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7e6c78d3-cf85-4546-b200-d4a4012c1901 None None] MFA Rules not processed for user `aac85b79585049608c74d181add6ed35`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:27.441004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7e6c78d3-cf85-4546-b200-d4a4012c1901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:27.480236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:27.528755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 961/1911] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:27.540463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d800-a6e2-43ee-9547-4de344ee2302 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:27.542683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4764d800-a6e2-43ee-9547-4de344ee2302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:27.615208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d800-a6e2-43ee-9547-4de344ee2302 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:27.618815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f406f-481c-49ed-87c5-d18c0abb1336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:27.619980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f406f-481c-49ed-87c5-d18c0abb1336 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:27.620330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f406f-481c-49ed-87c5-d18c0abb1336 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:27.620556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f406f-481c-49ed-87c5-d18c0abb1336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:27.683053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f406f-481c-49ed-87c5-d18c0abb1336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:27.684537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f406f-481c-49ed-87c5-d18c0abb1336 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:27.696213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 951/1912] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:31.206501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c78d3-cf85-4546-b200-d4a4012c1901 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:31.209462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7e6c78d3-cf85-4546-b200-d4a4012c1901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:31.243921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c78d3-cf85-4546-b200-d4a4012c1901 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:31.257428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09a46b-ae63-41a3-9a34-32e74b5f70f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:31.259795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09a46b-ae63-41a3-9a34-32e74b5f70f7 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:31.261659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09a46b-ae63-41a3-9a34-32e74b5f70f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:31.262208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09a46b-ae63-41a3-9a34-32e74b5f70f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:31.282252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09a46b-ae63-41a3-9a34-32e74b5f70f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:31.285418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09a46b-ae63-41a3-9a34-32e74b5f70f7 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:31.300889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 962/1913] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:37.690287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:INSERT=1,keystone:SELECT=50 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:41.294376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=20 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:52.967128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f406f-481c-49ed-87c5-d18c0abb1336 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:52.968552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-993f406f-481c-49ed-87c5-d18c0abb1336 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:53.087830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f406f-481c-49ed-87c5-d18c0abb1336 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:53.094880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:53.096118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:53.096673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:53.097191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] PATH_INFO: `/v3/users/d88f214d857c4b218f73207e8309a03c` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:53.099424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88f214d857c4b218f73207e8309a03c)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:53.101764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:53.187469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88f214d857c4b218f73207e8309a03c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:53.206541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88f214d857c4b218f73207e8309a03c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:53.217173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88f214d857c4b218f73207e8309a03c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:53.218176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88f214d857c4b218f73207e8309a03c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:53.226913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88f214d857c4b218f73207e8309a03c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:53.234361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 952/1914] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:52 2022] DELETE /identity/v3/users/d88f214d857c4b218f73207e8309a03c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:53.238786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf4ab5-5626-4b30-a01f-ca7e4f2afbe4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:53.239316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf4ab5-5626-4b30-a01f-ca7e4f2afbe4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:53.239316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf4ab5-5626-4b30-a01f-ca7e4f2afbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:53.257610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ebcf4ab5-5626-4b30-a01f-ca7e4f2afbe4 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:53.258836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ebcf4ab5-5626-4b30-a01f-ca7e4f2afbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:53.350173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 963/1915] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:53.361259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:53.363149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:53.467777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eda154-99f3-4b4c-8c2c-c0f44a8a34e4 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:53.472003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605eb56-1b70-404f-b15d-64edc967f454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:53.472921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605eb56-1b70-404f-b15d-64edc967f454 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:53.474076 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605eb56-1b70-404f-b15d-64edc967f454 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:53.474076 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605eb56-1b70-404f-b15d-64edc967f454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:53.499198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605eb56-1b70-404f-b15d-64edc967f454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:53.501067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605eb56-1b70-404f-b15d-64edc967f454 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:53.520670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 953/1916] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:54.226840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf4ab5-5626-4b30-a01f-ca7e4f2afbe4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:54.228676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ebcf4ab5-5626-4b30-a01f-ca7e4f2afbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:54.254169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf4ab5-5626-4b30-a01f-ca7e4f2afbe4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:54.279380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:54.280477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:54.280888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:54.281346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] PATH_INFO: `/v3/projects/d1df784ba9ff44689227952c58189999` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:54.284677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1df784ba9ff44689227952c58189999)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:54.286758 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:54.432003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1df784ba9ff44689227952c58189999', 'request_id': 'req-909bcd92-304a-4f09-b785-86a3af6ad12c'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:54.453261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1df784ba9ff44689227952c58189999', 'request_id': 'req-909bcd92-304a-4f09-b785-86a3af6ad12c'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:54.462921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1df784ba9ff44689227952c58189999', 'request_id': 'req-909bcd92-304a-4f09-b785-86a3af6ad12c'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:54.490349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] The token cache is being invalidate because project d1df784ba9ff44689227952c58189999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:54.490982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:54.491869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:54.504918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 964/1917] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:54 2022] DELETE /identity/v3/projects/d1df784ba9ff44689227952c58189999 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:54.509921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be612e-e17f-4b2c-b381-bc3e0e143f0f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:54.510398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be612e-e17f-4b2c-b381-bc3e0e143f0f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:54.510484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be612e-e17f-4b2c-b381-bc3e0e143f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:54.547645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f3be612e-e17f-4b2c-b381-bc3e0e143f0f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:54.548927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f3be612e-e17f-4b2c-b381-bc3e0e143f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:54.571068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:54.690149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:54.692449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:54.715960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 954/1918] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:54.725004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2010e-1f98-4da4-b345-6839f4a613e5 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:54.725326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2010e-1f98-4da4-b345-6839f4a613e5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:54.726074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2010e-1f98-4da4-b345-6839f4a613e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:54.773727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d5a2010e-1f98-4da4-b345-6839f4a613e5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:54.775981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d5a2010e-1f98-4da4-b345-6839f4a613e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:54.788931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 955/1919] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:54.794276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2010e-1f98-4da4-b345-6839f4a613e5 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:54.812619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2010e-1f98-4da4-b345-6839f4a613e5 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:54.818783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba4271-b55d-4e81-a128-237ff6c354be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:54.820488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba4271-b55d-4e81-a128-237ff6c354be admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:54.820960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba4271-b55d-4e81-a128-237ff6c354be admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:54.821409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba4271-b55d-4e81-a128-237ff6c354be admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:54.822324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba4271-b55d-4e81-a128-237ff6c354be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:54.822933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba4271-b55d-4e81-a128-237ff6c354be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:54.826894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba4271-b55d-4e81-a128-237ff6c354be admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:54.880518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 956/1920] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:49:54.885325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba4271-b55d-4e81-a128-237ff6c354be admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:54.892934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bcd92-304a-4f09-b785-86a3af6ad12c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:54.898500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:54.899531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:54.899901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:54.900099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] PATH_INFO: `/v3/users/0e39a6172b324166874d5086746b1e18` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:54.901389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e39a6172b324166874d5086746b1e18)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:54.904311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba4271-b55d-4e81-a128-237ff6c354be admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:54.904568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:54.909787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07f9c0-5605-477e-b062-db7ff18552d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:54.910862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07f9c0-5605-477e-b062-db7ff18552d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:54.911222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07f9c0-5605-477e-b062-db7ff18552d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:54.911575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07f9c0-5605-477e-b062-db7ff18552d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:54.913527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07f9c0-5605-477e-b062-db7ff18552d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:54.916330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07f9c0-5605-477e-b062-db7ff18552d4 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:54.961066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:54.994075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 957/1921] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:54 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:54.995496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07f9c0-5605-477e-b062-db7ff18552d4 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:55.000334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e39a6172b324166874d5086746b1e18'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:55.019629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e39a6172b324166874d5086746b1e18'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:55.020385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e39a6172b324166874d5086746b1e18'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:55.021867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e39a6172b324166874d5086746b1e18'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:55.032095 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-73f5d270-f1b5-44e3-adab-d3b94fd16602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e39a6172b324166874d5086746b1e18'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:55.052411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 965/1922] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:54 2022] DELETE /identity/v3/users/0e39a6172b324166874d5086746b1e18 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:55.057088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded20e6b-f837-4f2d-bb1b-248b70468e1b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:55.059267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded20e6b-f837-4f2d-bb1b-248b70468e1b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:55.059639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded20e6b-f837-4f2d-bb1b-248b70468e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:55.088619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ded20e6b-f837-4f2d-bb1b-248b70468e1b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:55.089991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ded20e6b-f837-4f2d-bb1b-248b70468e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:55.118535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07f9c0-5605-477e-b062-db7ff18552d4 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:55.171467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 966/1923] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:55.181723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded20e6b-f837-4f2d-bb1b-248b70468e1b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:55.183275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ded20e6b-f837-4f2d-bb1b-248b70468e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:55.281775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289241a0-4351-45a2-a871-3425b4149660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:55.282486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289241a0-4351-45a2-a871-3425b4149660 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:55.282779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289241a0-4351-45a2-a871-3425b4149660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:55.285047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289241a0-4351-45a2-a871-3425b4149660 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:55.285774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289241a0-4351-45a2-a871-3425b4149660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:55.293607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289241a0-4351-45a2-a871-3425b4149660 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:55.316468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded20e6b-f837-4f2d-bb1b-248b70468e1b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:55.325236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa8e73-09f0-4147-b771-d4de4b4baef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:55.326163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa8e73-09f0-4147-b771-d4de4b4baef6 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:55.326894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa8e73-09f0-4147-b771-d4de4b4baef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:55.327193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa8e73-09f0-4147-b771-d4de4b4baef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:55.347629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 958/1924] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:54 2022] POST /identity/v3/users => generated 292 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:49:55.349112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289241a0-4351-45a2-a871-3425b4149660 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:55.352617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-289241a0-4351-45a2-a871-3425b4149660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:55.358345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa8e73-09f0-4147-b771-d4de4b4baef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:55.361350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa8e73-09f0-4147-b771-d4de4b4baef6 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:55.388258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289241a0-4351-45a2-a871-3425b4149660 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:55.404253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3d282-a2eb-4daf-9362-a2020cac0994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:55.405238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3d282-a2eb-4daf-9362-a2020cac0994 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:55.405717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3d282-a2eb-4daf-9362-a2020cac0994 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:55.405943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3d282-a2eb-4daf-9362-a2020cac0994 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:55.406796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3d282-a2eb-4daf-9362-a2020cac0994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:55.407218 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 967/1925] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:55.408665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3d282-a2eb-4daf-9362-a2020cac0994 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:55.424627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 959/1926] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:55.427768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa8e73-09f0-4147-b771-d4de4b4baef6 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:55.439031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa8e73-09f0-4147-b771-d4de4b4baef6 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:55.447918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19c84f-2d0b-478f-8a8c-9a0e10bfa2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:55.449673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19c84f-2d0b-478f-8a8c-9a0e10bfa2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:55.449673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19c84f-2d0b-478f-8a8c-9a0e10bfa2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:55.450019 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19c84f-2d0b-478f-8a8c-9a0e10bfa2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:55.450860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19c84f-2d0b-478f-8a8c-9a0e10bfa2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:55.453121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19c84f-2d0b-478f-8a8c-9a0e10bfa2d9 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:55.473109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 968/1927] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:55.475238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3d282-a2eb-4daf-9362-a2020cac0994 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:55.489602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3d282-a2eb-4daf-9362-a2020cac0994 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:55.497284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3009dc4-b8a6-41e8-a737-a4db4880b0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:55.498304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3009dc4-b8a6-41e8-a737-a4db4880b0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:55.498659 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3009dc4-b8a6-41e8-a737-a4db4880b0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:55.499029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3009dc4-b8a6-41e8-a737-a4db4880b0ba admin admin] PATH_INFO: `/v3/projects/ccbe8abfccda48bc9aa0d201e2ef2a15/users/1c910e382192434aa80382a67a0bf5ad/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:55.507835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:55.538090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3009dc4-b8a6-41e8-a737-a4db4880b0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccbe8abfccda48bc9aa0d201e2ef2a15, user_id=1c910e382192434aa80382a67a0bf5ad, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:55.542351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3009dc4-b8a6-41e8-a737-a4db4880b0ba admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:55.564667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 960/1928] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:55 2022] PUT /identity/v3/projects/ccbe8abfccda48bc9aa0d201e2ef2a15/users/1c910e382192434aa80382a67a0bf5ad/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:55.576396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2db48-6068-415b-a206-5c0a04f1621a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:55.576958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2db48-6068-415b-a206-5c0a04f1621a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:55.579727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2db48-6068-415b-a206-5c0a04f1621a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:55.639913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2cc2db48-6068-415b-a206-5c0a04f1621a None None] MFA Rules not processed for user `1c910e382192434aa80382a67a0bf5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:55.645742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2cc2db48-6068-415b-a206-5c0a04f1621a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:55.705113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:55.796236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 969/1929] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:55.799039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3009dc4-b8a6-41e8-a737-a4db4880b0ba admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:56.003904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3009dc4-b8a6-41e8-a737-a4db4880b0ba admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:56.011875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4dca4-d3f5-4360-8f54-2033b5a93c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:56.012567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4dca4-d3f5-4360-8f54-2033b5a93c3d admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:56.012872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4dca4-d3f5-4360-8f54-2033b5a93c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:56.013147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4dca4-d3f5-4360-8f54-2033b5a93c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:56.014413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4dca4-d3f5-4360-8f54-2033b5a93c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:56.021709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4dca4-d3f5-4360-8f54-2033b5a93c3d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:56.100742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 961/1930] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:55 2022] POST /identity/v3/projects => generated 385 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:56.103765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2db48-6068-415b-a206-5c0a04f1621a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:56.122801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2db48-6068-415b-a206-5c0a04f1621a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:56.228492 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21c243-df10-4941-9cc8-841ca3a54c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:56.229312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21c243-df10-4941-9cc8-841ca3a54c87 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:56.229843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21c243-df10-4941-9cc8-841ca3a54c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:56.231303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21c243-df10-4941-9cc8-841ca3a54c87 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:56.232452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21c243-df10-4941-9cc8-841ca3a54c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:56.235829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21c243-df10-4941-9cc8-841ca3a54c87 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:56.286860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 970/1931] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:56 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:56.288279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4dca4-d3f5-4360-8f54-2033b5a93c3d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:56.302753 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4dca4-d3f5-4360-8f54-2033b5a93c3d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:56.310150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cebf9-b013-42f8-8915-60f1b36a9e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:56.314373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cebf9-b013-42f8-8915-60f1b36a9e1d admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:56.314693 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cebf9-b013-42f8-8915-60f1b36a9e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:56.315082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cebf9-b013-42f8-8915-60f1b36a9e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:56.315923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cebf9-b013-42f8-8915-60f1b36a9e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:56.320799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cebf9-b013-42f8-8915-60f1b36a9e1d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:56.357167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 962/1932] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:56.358984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21c243-df10-4941-9cc8-841ca3a54c87 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:56.373436 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21c243-df10-4941-9cc8-841ca3a54c87 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:56.388001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66125c90-314a-4864-830e-949b9c8dc3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:56.388742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66125c90-314a-4864-830e-949b9c8dc3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:56.389035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66125c90-314a-4864-830e-949b9c8dc3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:56.389319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66125c90-314a-4864-830e-949b9c8dc3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:56.390248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66125c90-314a-4864-830e-949b9c8dc3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:56.392161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66125c90-314a-4864-830e-949b9c8dc3c3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:56.408564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 971/1933] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:49:56.409892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cebf9-b013-42f8-8915-60f1b36a9e1d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:56.423440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cebf9-b013-42f8-8915-60f1b36a9e1d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:56.430423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eaa156-3f6b-45d0-9760-4bb87f4940ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:56.431942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eaa156-3f6b-45d0-9760-4bb87f4940ff admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:56.432250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eaa156-3f6b-45d0-9760-4bb87f4940ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:56.432736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eaa156-3f6b-45d0-9760-4bb87f4940ff admin admin] PATH_INFO: `/v3/projects/2bebe0a95fcf48d0969475b921e9d3c1/users/5a6053f66c5045e6990b62c73a7dcbd2/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:56.467623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eaa156-3f6b-45d0-9760-4bb87f4940ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bebe0a95fcf48d0969475b921e9d3c1, user_id=5a6053f66c5045e6990b62c73a7dcbd2, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:56.470734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eaa156-3f6b-45d0-9760-4bb87f4940ff admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:56.503214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 963/1934] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:56 2022] PUT /identity/v3/projects/2bebe0a95fcf48d0969475b921e9d3c1/users/5a6053f66c5045e6990b62c73a7dcbd2/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:56.515769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b79d9-d8a4-4b8c-828b-ea0aa5528047 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:56.516143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b79d9-d8a4-4b8c-828b-ea0aa5528047 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:56.516504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b79d9-d8a4-4b8c-828b-ea0aa5528047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:56.547968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a77b79d9-d8a4-4b8c-828b-ea0aa5528047 None None] MFA Rules not processed for user `5a6053f66c5045e6990b62c73a7dcbd2`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:56.549282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a77b79d9-d8a4-4b8c-828b-ea0aa5528047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:56.648974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eaa156-3f6b-45d0-9760-4bb87f4940ff admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:56.650065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 972/1935] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:56.651302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c8eaa156-3f6b-45d0-9760-4bb87f4940ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:56.654304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b79d9-d8a4-4b8c-828b-ea0aa5528047 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:56.707054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:56.745381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b79d9-d8a4-4b8c-828b-ea0aa5528047 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:56.759456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a0188-c76a-43e3-84d1-f25998bf54a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:56.760185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a0188-c76a-43e3-84d1-f25998bf54a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:56.760629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a0188-c76a-43e3-84d1-f25998bf54a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:56.760927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a0188-c76a-43e3-84d1-f25998bf54a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:56.762281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a0188-c76a-43e3-84d1-f25998bf54a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:56.764506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a0188-c76a-43e3-84d1-f25998bf54a0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:56.804343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eaa156-3f6b-45d0-9760-4bb87f4940ff admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:56.812780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 973/1936] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:56 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:49:56.814594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a0188-c76a-43e3-84d1-f25998bf54a0 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:56.818383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:56.819486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:56.819842 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:56.820074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] PATH_INFO: `/v3/projects/b5c1794ea4ab49fda088b08b2f63d03e` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:56.823133 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5c1794ea4ab49fda088b08b2f63d03e)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:56.826313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:56.883882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:56.899797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a0188-c76a-43e3-84d1-f25998bf54a0 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:56.901716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5c1794ea4ab49fda088b08b2f63d03e', 'request_id': 'req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:56.910679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5c1794ea4ab49fda088b08b2f63d03e', 'request_id': 'req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:56.922133 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5c1794ea4ab49fda088b08b2f63d03e', 'request_id': 'req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:56.945846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] The token cache is being invalidate because project b5c1794ea4ab49fda088b08b2f63d03e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:56.946473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:56.947156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:56.950092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 964/1937] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:56 2022] DELETE /identity/v3/projects/b5c1794ea4ab49fda088b08b2f63d03e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:56.954054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:56.956248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:56.982254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de039643-6d5d-438f-8517-efcaac09d3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:56.982254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de039643-6d5d-438f-8517-efcaac09d3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:56.982254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de039643-6d5d-438f-8517-efcaac09d3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:56.982641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de039643-6d5d-438f-8517-efcaac09d3b0 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:56.983950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de039643-6d5d-438f-8517-efcaac09d3b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:56.987438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de039643-6d5d-438f-8517-efcaac09d3b0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:57.025003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 974/1938] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:56 2022] POST /identity/v3/users => generated 298 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:57.030487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de039643-6d5d-438f-8517-efcaac09d3b0 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:57.041864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-de039643-6d5d-438f-8517-efcaac09d3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:57.051871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de039643-6d5d-438f-8517-efcaac09d3b0 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:57.058881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77663316-fbcd-42c5-8a1e-5473bd5a6783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:57.059913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77663316-fbcd-42c5-8a1e-5473bd5a6783 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:57.060302 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77663316-fbcd-42c5-8a1e-5473bd5a6783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:57.060757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77663316-fbcd-42c5-8a1e-5473bd5a6783 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:57.061879 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77663316-fbcd-42c5-8a1e-5473bd5a6783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:57.064435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77663316-fbcd-42c5-8a1e-5473bd5a6783 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:57.067266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bed1e-0966-46f7-aa4c-7fe82b9b2efb admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:57.072213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:57.073136 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:57.073422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:57.074224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] PATH_INFO: `/v3/users/a06122bd1b3e4eb386f58956e19781a2` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:57.075777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06122bd1b3e4eb386f58956e19781a2)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:57.077756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:57.080493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 975/1939] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:57.084761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77663316-fbcd-42c5-8a1e-5473bd5a6783 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:57.107000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77663316-fbcd-42c5-8a1e-5473bd5a6783 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:57.118526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102792f6-6488-4b64-9c05-e8c74caabc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:57.119585 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102792f6-6488-4b64-9c05-e8c74caabc83 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:57.119874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102792f6-6488-4b64-9c05-e8c74caabc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:57.120154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102792f6-6488-4b64-9c05-e8c74caabc83 admin admin] PATH_INFO: `/v3/projects/2e7f8df56c274c2d8092b7fc1d850ffb/users/8918f136484e46eba5ef77ed5abdccb1/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:57.137906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102792f6-6488-4b64-9c05-e8c74caabc83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7f8df56c274c2d8092b7fc1d850ffb, user_id=8918f136484e46eba5ef77ed5abdccb1, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:57.140569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102792f6-6488-4b64-9c05-e8c74caabc83 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:57.147668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06122bd1b3e4eb386f58956e19781a2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.157003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 976/1940] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:57 2022] PUT /identity/v3/projects/2e7f8df56c274c2d8092b7fc1d850ffb/users/8918f136484e46eba5ef77ed5abdccb1/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:57.160962 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102792f6-6488-4b64-9c05-e8c74caabc83 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:57.164749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06122bd1b3e4eb386f58956e19781a2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.178126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06122bd1b3e4eb386f58956e19781a2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.178887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06122bd1b3e4eb386f58956e19781a2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.183692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:57.193438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06122bd1b3e4eb386f58956e19781a2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.205249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 965/1941] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:56 2022] DELETE /identity/v3/users/a06122bd1b3e4eb386f58956e19781a2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:57.244080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102792f6-6488-4b64-9c05-e8c74caabc83 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:57.314434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a71931-f670-4799-865b-789809e27522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:57.315419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a71931-f670-4799-865b-789809e27522 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:57.315533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a71931-f670-4799-865b-789809e27522 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:57.315817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a71931-f670-4799-865b-789809e27522 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:57.316694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a71931-f670-4799-865b-789809e27522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:57.318377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a71931-f670-4799-865b-789809e27522 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:57.331817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 977/1942] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:57.337106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:57.338685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:57.358889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea649a-cd4a-4829-a6d2-97da34254d5b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:57.363861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de287fc6-01d0-4723-845c-70cd0c9a4007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:57.364504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de287fc6-01d0-4723-845c-70cd0c9a4007 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:57.364896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de287fc6-01d0-4723-845c-70cd0c9a4007 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:57.365245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de287fc6-01d0-4723-845c-70cd0c9a4007 admin admin] PATH_INFO: `/v3/projects/2e7f8df56c274c2d8092b7fc1d850ffb/users/8918f136484e46eba5ef77ed5abdccb1/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:57.367468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de287fc6-01d0-4723-845c-70cd0c9a4007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7f8df56c274c2d8092b7fc1d850ffb, user_id=8918f136484e46eba5ef77ed5abdccb1, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:57.370401 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de287fc6-01d0-4723-845c-70cd0c9a4007 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:57.395236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee1cf3-833e-42f3-9607-cc81c6b20d2a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:57.395721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee1cf3-833e-42f3-9607-cc81c6b20d2a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:57.396722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee1cf3-833e-42f3-9607-cc81c6b20d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:57.398015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 966/1943] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:57 2022] PUT /identity/v3/projects/2e7f8df56c274c2d8092b7fc1d850ffb/users/8918f136484e46eba5ef77ed5abdccb1/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:57.434372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-72ee1cf3-833e-42f3-9607-cc81c6b20d2a None None] MFA Rules not processed for user `8918f136484e46eba5ef77ed5abdccb1`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:57.436012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-72ee1cf3-833e-42f3-9607-cc81c6b20d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:57.534984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 978/1944] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:57.547816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de287fc6-01d0-4723-845c-70cd0c9a4007 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:57.549632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-de287fc6-01d0-4723-845c-70cd0c9a4007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:57.605448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:57.633686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee1cf3-833e-42f3-9607-cc81c6b20d2a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:57.635243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-72ee1cf3-833e-42f3-9607-cc81c6b20d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:57.669649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de287fc6-01d0-4723-845c-70cd0c9a4007 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:57.673967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e56d2e-401a-4529-8d1e-276ea7551f8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:57.674801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e56d2e-401a-4529-8d1e-276ea7551f8d service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:57.675213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e56d2e-401a-4529-8d1e-276ea7551f8d service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:57.675594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e56d2e-401a-4529-8d1e-276ea7551f8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:57.677777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d5e56d2e-401a-4529-8d1e-276ea7551f8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:57.742096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:57.774307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee1cf3-833e-42f3-9607-cc81c6b20d2a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:57.780316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e56d2e-401a-4529-8d1e-276ea7551f8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:57.780868 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:57.781810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:57.782161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:57.782266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e56d2e-401a-4529-8d1e-276ea7551f8d service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:57.782539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] PATH_INFO: `/v3/projects/11f0a7d143da4eb3bdefd1b805da28f1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:57.784055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f0a7d143da4eb3bdefd1b805da28f1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:57.786237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:57.797019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 967/1945] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:57.900664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f0a7d143da4eb3bdefd1b805da28f1', 'request_id': 'req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.916564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f0a7d143da4eb3bdefd1b805da28f1', 'request_id': 'req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.928437 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f0a7d143da4eb3bdefd1b805da28f1', 'request_id': 'req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.960959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] The token cache is being invalidate because project 11f0a7d143da4eb3bdefd1b805da28f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:49:57.961275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.965102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-7aaf1d3a-672d-49a4-a91d-34e61ffd79c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:57.970129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 979/1946] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:49:57 2022] DELETE /identity/v3/projects/11f0a7d143da4eb3bdefd1b805da28f1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:57.971888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd727ccb-4528-4e7a-aeac-28e3a322c205 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:57.972306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd727ccb-4528-4e7a-aeac-28e3a322c205 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:57.972742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd727ccb-4528-4e7a-aeac-28e3a322c205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.016267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-dd727ccb-4528-4e7a-aeac-28e3a322c205 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:58.018651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-dd727ccb-4528-4e7a-aeac-28e3a322c205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:58.141164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 968/1947] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:58.144209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833aaa8-18a7-4f85-b505-b3cee22fb5b3 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.144569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833aaa8-18a7-4f85-b505-b3cee22fb5b3 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.145104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833aaa8-18a7-4f85-b505-b3cee22fb5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.171061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6833aaa8-18a7-4f85-b505-b3cee22fb5b3 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:58.172707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6833aaa8-18a7-4f85-b505-b3cee22fb5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:58.192349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 980/1948] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:58.196884 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd727ccb-4528-4e7a-aeac-28e3a322c205 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:58.210117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd727ccb-4528-4e7a-aeac-28e3a322c205 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:58.226346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228880a8-7081-4c7e-b4d6-c4fba10bf006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:58.227178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228880a8-7081-4c7e-b4d6-c4fba10bf006 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.227556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228880a8-7081-4c7e-b4d6-c4fba10bf006 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.227963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228880a8-7081-4c7e-b4d6-c4fba10bf006 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.228890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228880a8-7081-4c7e-b4d6-c4fba10bf006 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:49:58.229310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228880a8-7081-4c7e-b4d6-c4fba10bf006 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:58.234156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228880a8-7081-4c7e-b4d6-c4fba10bf006 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:58.271853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 969/1949] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:49:58.278236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833aaa8-18a7-4f85-b505-b3cee22fb5b3 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:58.288372 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833aaa8-18a7-4f85-b505-b3cee22fb5b3 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:58.295272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b6ee3-c99b-40de-b3b4-4bace4f18d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:58.295595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b6ee3-c99b-40de-b3b4-4bace4f18d38 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.295885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b6ee3-c99b-40de-b3b4-4bace4f18d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.296167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b6ee3-c99b-40de-b3b4-4bace4f18d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.296969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b6ee3-c99b-40de-b3b4-4bace4f18d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:58.299098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b6ee3-c99b-40de-b3b4-4bace4f18d38 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:58.350457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 981/1950] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:58 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:49:58.354148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228880a8-7081-4c7e-b4d6-c4fba10bf006 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:58.369867 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228880a8-7081-4c7e-b4d6-c4fba10bf006 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:58.475913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab24ec-c5d0-4c8d-94cc-cbf1d8d497cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:58.476700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab24ec-c5d0-4c8d-94cc-cbf1d8d497cd admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.476992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab24ec-c5d0-4c8d-94cc-cbf1d8d497cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.477304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab24ec-c5d0-4c8d-94cc-cbf1d8d497cd admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.478165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab24ec-c5d0-4c8d-94cc-cbf1d8d497cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:58.480255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab24ec-c5d0-4c8d-94cc-cbf1d8d497cd admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:58.502628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:58.512775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 970/1951] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:58 2022] POST /identity/v3/users => generated 284 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:49:58.514264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b6ee3-c99b-40de-b3b4-4bace4f18d38 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:58.525851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b6ee3-c99b-40de-b3b4-4bace4f18d38 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:58.531659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbf8b6-6f80-4ab1-9340-6e794ffedba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:58.532346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbf8b6-6f80-4ab1-9340-6e794ffedba5 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.532664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbf8b6-6f80-4ab1-9340-6e794ffedba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.532943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbf8b6-6f80-4ab1-9340-6e794ffedba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.533685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbf8b6-6f80-4ab1-9340-6e794ffedba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:58.535339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbf8b6-6f80-4ab1-9340-6e794ffedba5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:58.547068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 982/1952] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:58.550968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab24ec-c5d0-4c8d-94cc-cbf1d8d497cd admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:58.562389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab24ec-c5d0-4c8d-94cc-cbf1d8d497cd admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:58.567723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1eb9c-b763-4bcb-a41e-39ef04449f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:58.568448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1eb9c-b763-4bcb-a41e-39ef04449f2a admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.568897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1eb9c-b763-4bcb-a41e-39ef04449f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.569371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1eb9c-b763-4bcb-a41e-39ef04449f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.570065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1eb9c-b763-4bcb-a41e-39ef04449f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:58.571697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1eb9c-b763-4bcb-a41e-39ef04449f2a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:58.585513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 971/1953] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:58.589946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbf8b6-6f80-4ab1-9340-6e794ffedba5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:58.600553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbf8b6-6f80-4ab1-9340-6e794ffedba5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:58.606547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc215c-2251-49dc-a7e9-e2682e6b2392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:58.608028 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc215c-2251-49dc-a7e9-e2682e6b2392 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.608321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc215c-2251-49dc-a7e9-e2682e6b2392 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.608634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc215c-2251-49dc-a7e9-e2682e6b2392 admin admin] PATH_INFO: `/v3/projects/13d4a4bf29e6431cb6c1f9b91b34a3e2/users/5327cb42ab4a40c88493279f5bb85505/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.627542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc215c-2251-49dc-a7e9-e2682e6b2392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d4a4bf29e6431cb6c1f9b91b34a3e2, user_id=5327cb42ab4a40c88493279f5bb85505, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:58.631298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc215c-2251-49dc-a7e9-e2682e6b2392 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:58.648467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 983/1954] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:58 2022] PUT /identity/v3/projects/13d4a4bf29e6431cb6c1f9b91b34a3e2/users/5327cb42ab4a40c88493279f5bb85505/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:58.660916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089ab83-b6fe-4747-9545-ccf6e4880bea None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.661110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089ab83-b6fe-4747-9545-ccf6e4880bea None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.661349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089ab83-b6fe-4747-9545-ccf6e4880bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.705298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860f9d-eb33-482d-8050-6204b5e409c5 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.705775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860f9d-eb33-482d-8050-6204b5e409c5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.706297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860f9d-eb33-482d-8050-6204b5e409c5 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.707199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-3089ab83-b6fe-4747-9545-ccf6e4880bea None None] MFA Rules not processed for user `5327cb42ab4a40c88493279f5bb85505`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:58.709058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3089ab83-b6fe-4747-9545-ccf6e4880bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:58.709212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 984/1955] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:49:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:49:58.715858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6147c56-5245-4a29-8185-16754859d605 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.716632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6147c56-5245-4a29-8185-16754859d605 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.716632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6147c56-5245-4a29-8185-16754859d605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.739835 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-c6147c56-5245-4a29-8185-16754859d605 None None] MFA Rules not processed for user `a243d444b7ae487cb0331523d4404d19`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:58.741272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c6147c56-5245-4a29-8185-16754859d605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:58.813416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 972/1956] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:58.817413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089ab83-b6fe-4747-9545-ccf6e4880bea None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:58.855970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 985/1957] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:49:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:49:58.862714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6147c56-5245-4a29-8185-16754859d605 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:58.864346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c6147c56-5245-4a29-8185-16754859d605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:58.882518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6147c56-5245-4a29-8185-16754859d605 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:58.889205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf78f5-d00e-479e-b9ae-a1ac39e1273b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:58.890388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf78f5-d00e-479e-b9ae-a1ac39e1273b service placement] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.890993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf78f5-d00e-479e-b9ae-a1ac39e1273b service placement] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.891429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf78f5-d00e-479e-b9ae-a1ac39e1273b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.899824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089ab83-b6fe-4747-9545-ccf6e4880bea None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:58.904784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf78f5-d00e-479e-b9ae-a1ac39e1273b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:58.906322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdba7a3-1b31-429b-bb59-781b58766739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:58.907061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdba7a3-1b31-429b-bb59-781b58766739 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:58.907376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf78f5-d00e-479e-b9ae-a1ac39e1273b service placement] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:58.907541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdba7a3-1b31-429b-bb59-781b58766739 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:58.907974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdba7a3-1b31-429b-bb59-781b58766739 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:58.908880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdba7a3-1b31-429b-bb59-781b58766739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:58.911026 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdba7a3-1b31-429b-bb59-781b58766739 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:58.913001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:58.923159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 986/1958] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:49:58.953010 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 973/1959] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:49:58 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:49:58.957551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf78f5-d00e-479e-b9ae-a1ac39e1273b service placement] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:58.975017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf78f5-d00e-479e-b9ae-a1ac39e1273b service placement] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.045419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44b99d-3d94-4801-b538-0b5850259ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.046354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44b99d-3d94-4801-b538-0b5850259ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.047171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44b99d-3d94-4801-b538-0b5850259ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.047171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44b99d-3d94-4801-b538-0b5850259ef7 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.047692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44b99d-3d94-4801-b538-0b5850259ef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.049607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44b99d-3d94-4801-b538-0b5850259ef7 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.071808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 987/1960] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:49:58 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:49:59.076038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdba7a3-1b31-429b-bb59-781b58766739 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:59.089935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdba7a3-1b31-429b-bb59-781b58766739 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.094816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ca1db-1a39-46fd-99c0-2294f051d6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.095502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ca1db-1a39-46fd-99c0-2294f051d6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.095797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ca1db-1a39-46fd-99c0-2294f051d6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.096060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ca1db-1a39-46fd-99c0-2294f051d6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.096844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ca1db-1a39-46fd-99c0-2294f051d6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.098460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ca1db-1a39-46fd-99c0-2294f051d6a3 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.110372 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 974/1961] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:59.114519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44b99d-3d94-4801-b538-0b5850259ef7 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:59.125005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44b99d-3d94-4801-b538-0b5850259ef7 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.130048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd37db2-1a80-4786-b681-f87d19a4d418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.130979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd37db2-1a80-4786-b681-f87d19a4d418 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.131265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd37db2-1a80-4786-b681-f87d19a4d418 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.131542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd37db2-1a80-4786-b681-f87d19a4d418 admin admin] PATH_INFO: `/v3/projects/eec7458011ba4c0b939ab09a9ee99aac/users/63614e3fd48e4dfba532efe78e5c7797/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.150208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd37db2-1a80-4786-b681-f87d19a4d418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec7458011ba4c0b939ab09a9ee99aac, user_id=63614e3fd48e4dfba532efe78e5c7797, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.153663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd37db2-1a80-4786-b681-f87d19a4d418 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.170189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 988/1962] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:59 2022] PUT /identity/v3/projects/eec7458011ba4c0b939ab09a9ee99aac/users/63614e3fd48e4dfba532efe78e5c7797/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:59.173858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ca1db-1a39-46fd-99c0-2294f051d6a3 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:59.185857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ca1db-1a39-46fd-99c0-2294f051d6a3 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.258486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:59.270636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e679fc-945c-48bf-8467-bb72bd38fa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.271680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e679fc-945c-48bf-8467-bb72bd38fa40 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.272520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e679fc-945c-48bf-8467-bb72bd38fa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.273205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e679fc-945c-48bf-8467-bb72bd38fa40 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.273959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e679fc-945c-48bf-8467-bb72bd38fa40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.275848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e679fc-945c-48bf-8467-bb72bd38fa40 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.287901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 975/1963] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:49:59.291885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd37db2-1a80-4786-b681-f87d19a4d418 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:59.305189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd37db2-1a80-4786-b681-f87d19a4d418 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.310732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444152a7-7161-43da-9111-18d6a30be5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.311845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444152a7-7161-43da-9111-18d6a30be5e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.312221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444152a7-7161-43da-9111-18d6a30be5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.313585 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444152a7-7161-43da-9111-18d6a30be5e2 admin admin] PATH_INFO: `/v3/projects/eec7458011ba4c0b939ab09a9ee99aac/users/63614e3fd48e4dfba532efe78e5c7797/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.315076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444152a7-7161-43da-9111-18d6a30be5e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec7458011ba4c0b939ab09a9ee99aac, user_id=63614e3fd48e4dfba532efe78e5c7797, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.317949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444152a7-7161-43da-9111-18d6a30be5e2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.330993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 989/1964] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:49:59 2022] PUT /identity/v3/projects/eec7458011ba4c0b939ab09a9ee99aac/users/63614e3fd48e4dfba532efe78e5c7797/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:49:59.343986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04d84-edab-44a2-a451-f9252503adcb None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.344349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04d84-edab-44a2-a451-f9252503adcb None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.344852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04d84-edab-44a2-a451-f9252503adcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.374733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8bb04d84-edab-44a2-a451-f9252503adcb None None] MFA Rules not processed for user `63614e3fd48e4dfba532efe78e5c7797`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:59.376061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8bb04d84-edab-44a2-a451-f9252503adcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:59.402709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc91e1-f41e-4899-852b-4bf359d9a398 None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.402975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc91e1-f41e-4899-852b-4bf359d9a398 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.403270 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc91e1-f41e-4899-852b-4bf359d9a398 None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.406966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 990/1965] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:49:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:49:59.427000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51855453-1c6e-4577-af4c-ff1e71e1d520 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.427284 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51855453-1c6e-4577-af4c-ff1e71e1d520 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.427582 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51855453-1c6e-4577-af4c-ff1e71e1d520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.459662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-51855453-1c6e-4577-af4c-ff1e71e1d520 None None] MFA Rules not processed for user `a243d444b7ae487cb0331523d4404d19`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:49:59.461110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-51855453-1c6e-4577-af4c-ff1e71e1d520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:59.476048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 976/1966] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:49:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:59.489304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb04d84-edab-44a2-a451-f9252503adcb None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:59.491837 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8bb04d84-edab-44a2-a451-f9252503adcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:59.580219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb04d84-edab-44a2-a451-f9252503adcb None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.583993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 991/1967] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:49:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:49:59.584880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51855453-1c6e-4577-af4c-ff1e71e1d520 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:59.586418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-51855453-1c6e-4577-af4c-ff1e71e1d520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:59.588654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8d7f6-5ce8-4c6a-993a-2fbf40f8ff3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.589471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8d7f6-5ce8-4c6a-993a-2fbf40f8ff3b service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.589765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8d7f6-5ce8-4c6a-993a-2fbf40f8ff3b service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.590178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8d7f6-5ce8-4c6a-993a-2fbf40f8ff3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.606018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8d7f6-5ce8-4c6a-993a-2fbf40f8ff3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.607553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8d7f6-5ce8-4c6a-993a-2fbf40f8ff3b service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.617817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 977/1968] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:59.618671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8d7f6-5ce8-4c6a-993a-2fbf40f8ff3b service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:59.630611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8d7f6-5ce8-4c6a-993a-2fbf40f8ff3b service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.636050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc1b90-e81e-4f9d-8bb1-1605f5d736de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.637372 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc1b90-e81e-4f9d-8bb1-1605f5d736de service placement] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.637756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc1b90-e81e-4f9d-8bb1-1605f5d736de service placement] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.638382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc1b90-e81e-4f9d-8bb1-1605f5d736de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.642633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:59.648802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc1b90-e81e-4f9d-8bb1-1605f5d736de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.650461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc1b90-e81e-4f9d-8bb1-1605f5d736de service placement] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.661393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51855453-1c6e-4577-af4c-ff1e71e1d520 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.662118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 978/1969] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:59.665765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc1b90-e81e-4f9d-8bb1-1605f5d736de service placement] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:59.666782 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.668323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.668468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-eadc1b90-e81e-4f9d-8bb1-1605f5d736de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:59.668805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.669247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] PATH_INFO: `/v3/users/dedcdb7aedf642e8b19addcb8c268cb8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.670601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dedcdb7aedf642e8b19addcb8c268cb8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.672755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.682028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc1b90-e81e-4f9d-8bb1-1605f5d736de service placement] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.685985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2e982-5e58-4a7c-ba46-f399a12019fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.686760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2e982-5e58-4a7c-ba46-f399a12019fc service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.687050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2e982-5e58-4a7c-ba46-f399a12019fc service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.687333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2e982-5e58-4a7c-ba46-f399a12019fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.689432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c5f2e982-5e58-4a7c-ba46-f399a12019fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:59.737315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dedcdb7aedf642e8b19addcb8c268cb8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:59.751861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dedcdb7aedf642e8b19addcb8c268cb8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:59.752783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dedcdb7aedf642e8b19addcb8c268cb8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:59.753506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dedcdb7aedf642e8b19addcb8c268cb8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:59.761879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2e982-5e58-4a7c-ba46-f399a12019fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.763616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2e982-5e58-4a7c-ba46-f399a12019fc service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.766129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dedcdb7aedf642e8b19addcb8c268cb8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:49:59.775043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 979/1970] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:49:59.781558 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2e982-5e58-4a7c-ba46-f399a12019fc service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:49:59.785876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:49:59.788118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 992/1971] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:49:59 2022] DELETE /identity/v3/users/dedcdb7aedf642e8b19addcb8c268cb8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:49:59.794262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2e982-5e58-4a7c-ba46-f399a12019fc service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:49:59.874449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad58a6-5008-47e2-96a6-1f8234364f35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:49:59.875219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad58a6-5008-47e2-96a6-1f8234364f35 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:49:59.875565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad58a6-5008-47e2-96a6-1f8234364f35 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:49:59.876144 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad58a6-5008-47e2-96a6-1f8234364f35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:49:59.878150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-03ad58a6-5008-47e2-96a6-1f8234364f35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:49:59.956207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad58a6-5008-47e2-96a6-1f8234364f35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:49:59.957912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad58a6-5008-47e2-96a6-1f8234364f35 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:49:59.970125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 980/1972] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:49:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:00.005928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:00.007428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:00.086401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2881b-4519-4b64-a6ef-d60a857cae0e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:00.091837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:00.092827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:00.093196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:00.093551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] PATH_INFO: `/v3/projects/56c4740ad6074a03b54fa9c6bc93ffd8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:00.094954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56c4740ad6074a03b54fa9c6bc93ffd8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:00.096931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:00.158811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56c4740ad6074a03b54fa9c6bc93ffd8', 'request_id': 'req-a57beb88-accd-40e3-a252-ba7675d14632'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.166104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56c4740ad6074a03b54fa9c6bc93ffd8', 'request_id': 'req-a57beb88-accd-40e3-a252-ba7675d14632'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.173451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56c4740ad6074a03b54fa9c6bc93ffd8', 'request_id': 'req-a57beb88-accd-40e3-a252-ba7675d14632'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.179460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:00.190452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] The token cache is being invalidate because project 56c4740ad6074a03b54fa9c6bc93ffd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:00.190452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.190914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.193314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 993/1973] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:00 2022] DELETE /identity/v3/projects/56c4740ad6074a03b54fa9c6bc93ffd8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:00.198060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad58a6-5008-47e2-96a6-1f8234364f35 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:00.215891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad58a6-5008-47e2-96a6-1f8234364f35 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:00.309311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:00.310213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:00.310501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:00.311616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] PATH_INFO: `/v3/users/aac85b79585049608c74d181add6ed35` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:00.313662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aac85b79585049608c74d181add6ed35)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:00.317993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:00.407332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aac85b79585049608c74d181add6ed35'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.426478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:00.426478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aac85b79585049608c74d181add6ed35'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.433822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aac85b79585049608c74d181add6ed35'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.434694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aac85b79585049608c74d181add6ed35'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.443489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aac85b79585049608c74d181add6ed35'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.450531 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 981/1974] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:00 2022] DELETE /identity/v3/users/aac85b79585049608c74d181add6ed35 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:00.507700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:00.509293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:00.569339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57beb88-accd-40e3-a252-ba7675d14632 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:00.619459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9722b-c14e-4f97-9e94-fcfadda5867e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:00.620226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9722b-c14e-4f97-9e94-fcfadda5867e service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:00.621150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9722b-c14e-4f97-9e94-fcfadda5867e service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:00.621998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9722b-c14e-4f97-9e94-fcfadda5867e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:00.623992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-41e9722b-c14e-4f97-9e94-fcfadda5867e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:00.630655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:00.632266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:00.697357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9722b-c14e-4f97-9e94-fcfadda5867e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:00.699495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9722b-c14e-4f97-9e94-fcfadda5867e service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:00.708027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf51a3-46e8-4ab4-8f7d-0bf60064ecc7 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:00.710771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 994/1975] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:00.717625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:00.718571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:00.718929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:00.719320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] PATH_INFO: `/v3/projects/345f1057c6ad4abe90b005c974378e82` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:00.720617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=345f1057c6ad4abe90b005c974378e82)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:00.722658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:00.778170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345f1057c6ad4abe90b005c974378e82', 'request_id': 'req-67551325-2f84-4857-8515-363a19c0bc8d'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.784692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345f1057c6ad4abe90b005c974378e82', 'request_id': 'req-67551325-2f84-4857-8515-363a19c0bc8d'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.794957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345f1057c6ad4abe90b005c974378e82', 'request_id': 'req-67551325-2f84-4857-8515-363a19c0bc8d'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.807670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] The token cache is being invalidate because project 345f1057c6ad4abe90b005c974378e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:00.808003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.808946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-67551325-2f84-4857-8515-363a19c0bc8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:00.810854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 982/1976] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:00 2022] DELETE /identity/v3/projects/345f1057c6ad4abe90b005c974378e82 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:00.816898 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c271b6a-b9b0-4d16-8901-903677b8c88d None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:00.817051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c271b6a-b9b0-4d16-8901-903677b8c88d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:00.817338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c271b6a-b9b0-4d16-8901-903677b8c88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:00.844856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1c271b6a-b9b0-4d16-8901-903677b8c88d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:00.846539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1c271b6a-b9b0-4d16-8901-903677b8c88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:00.916757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 995/1977] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:00.925326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91753eb9-54ad-4636-942c-0f4c7b8ba836 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:00.926295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91753eb9-54ad-4636-942c-0f4c7b8ba836 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:00.927658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91753eb9-54ad-4636-942c-0f4c7b8ba836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:00.952650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-91753eb9-54ad-4636-942c-0f4c7b8ba836 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:00.953893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-91753eb9-54ad-4636-942c-0f4c7b8ba836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:00.963189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 983/1978] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:00.968632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c271b6a-b9b0-4d16-8901-903677b8c88d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:00.973811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:00.980796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c271b6a-b9b0-4d16-8901-903677b8c88d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:00.986238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b546b9-0aac-48da-8302-5fea440350ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:00.986943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b546b9-0aac-48da-8302-5fea440350ee admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:00.987231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b546b9-0aac-48da-8302-5fea440350ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:00.987512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b546b9-0aac-48da-8302-5fea440350ee admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:00.988254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b546b9-0aac-48da-8302-5fea440350ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:00.988565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b546b9-0aac-48da-8302-5fea440350ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:00.990321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b546b9-0aac-48da-8302-5fea440350ee admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:01.003743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 996/1979] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:50:01.008530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91753eb9-54ad-4636-942c-0f4c7b8ba836 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:01.021035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91753eb9-54ad-4636-942c-0f4c7b8ba836 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:01.030943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135115c-a38e-4f11-a730-b44aa2ed3e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:01.031621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135115c-a38e-4f11-a730-b44aa2ed3e64 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:01.031903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135115c-a38e-4f11-a730-b44aa2ed3e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:01.032176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135115c-a38e-4f11-a730-b44aa2ed3e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:01.032913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135115c-a38e-4f11-a730-b44aa2ed3e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:01.034822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135115c-a38e-4f11-a730-b44aa2ed3e64 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:01.062982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 984/1980] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:01 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:01.067321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b546b9-0aac-48da-8302-5fea440350ee admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:01.078312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b546b9-0aac-48da-8302-5fea440350ee admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:01.134795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee2387-6105-41a9-83e4-49b47986fb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:01.135468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee2387-6105-41a9-83e4-49b47986fb62 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:01.135755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee2387-6105-41a9-83e4-49b47986fb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:01.136035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee2387-6105-41a9-83e4-49b47986fb62 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:01.137275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee2387-6105-41a9-83e4-49b47986fb62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:01.139203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee2387-6105-41a9-83e4-49b47986fb62 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:01.160493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 997/1981] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:01 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:01.164611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135115c-a38e-4f11-a730-b44aa2ed3e64 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:01.175411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135115c-a38e-4f11-a730-b44aa2ed3e64 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:01.180038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abceda0-c417-4336-adfc-e5adb1a3d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:01.180797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abceda0-c417-4336-adfc-e5adb1a3d938 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:01.181082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abceda0-c417-4336-adfc-e5adb1a3d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:01.181358 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abceda0-c417-4336-adfc-e5adb1a3d938 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:01.182084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abceda0-c417-4336-adfc-e5adb1a3d938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:01.183673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abceda0-c417-4336-adfc-e5adb1a3d938 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:01.193840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 985/1982] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:01.197622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee2387-6105-41a9-83e4-49b47986fb62 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:01.208189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee2387-6105-41a9-83e4-49b47986fb62 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:01.213215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d3fa-a41c-4d2e-9566-40785316ba2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:01.213983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d3fa-a41c-4d2e-9566-40785316ba2d admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:01.214932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d3fa-a41c-4d2e-9566-40785316ba2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:01.215406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d3fa-a41c-4d2e-9566-40785316ba2d admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:01.216759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d3fa-a41c-4d2e-9566-40785316ba2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:01.219103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d3fa-a41c-4d2e-9566-40785316ba2d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:01.231789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 998/1983] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:01.236312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abceda0-c417-4336-adfc-e5adb1a3d938 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:01.247202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abceda0-c417-4336-adfc-e5adb1a3d938 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:01.252040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cf3b2-c7ed-4e2e-b920-bc3c8d27f706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:01.252947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cf3b2-c7ed-4e2e-b920-bc3c8d27f706 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:01.253225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cf3b2-c7ed-4e2e-b920-bc3c8d27f706 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:01.253494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cf3b2-c7ed-4e2e-b920-bc3c8d27f706 admin admin] PATH_INFO: `/v3/projects/d169407e7e614960aadfbc8542b73cc6/users/1500a7221a444ac6966c2c1d1335b902/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:01.268159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cf3b2-c7ed-4e2e-b920-bc3c8d27f706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d169407e7e614960aadfbc8542b73cc6, user_id=1500a7221a444ac6966c2c1d1335b902, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:01.270828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cf3b2-c7ed-4e2e-b920-bc3c8d27f706 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:01.284481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 986/1984] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:01 2022] PUT /identity/v3/projects/d169407e7e614960aadfbc8542b73cc6/users/1500a7221a444ac6966c2c1d1335b902/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:01.295991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f9f6d-a41a-41a5-966a-40e5920f3f0b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:01.296312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f9f6d-a41a-41a5-966a-40e5920f3f0b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:01.296689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f9f6d-a41a-41a5-966a-40e5920f3f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:01.324240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ff1f9f6d-a41a-41a5-966a-40e5920f3f0b None None] MFA Rules not processed for user `1500a7221a444ac6966c2c1d1335b902`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:01.325481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ff1f9f6d-a41a-41a5-966a-40e5920f3f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:01.391940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 999/1985] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:01.404429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cf3b2-c7ed-4e2e-b920-bc3c8d27f706 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:01.406194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0b7cf3b2-c7ed-4e2e-b920-bc3c8d27f706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:01.411742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:01.481889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cf3b2-c7ed-4e2e-b920-bc3c8d27f706 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:01.490571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84800d-3662-4e72-b07b-833a73ec5a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:01.491385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84800d-3662-4e72-b07b-833a73ec5a4f service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:01.491727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84800d-3662-4e72-b07b-833a73ec5a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:01.492101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84800d-3662-4e72-b07b-833a73ec5a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:01.505503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84800d-3662-4e72-b07b-833a73ec5a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:01.507288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84800d-3662-4e72-b07b-833a73ec5a4f service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:01.518148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 987/1986] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:03.633873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f9f6d-a41a-41a5-966a-40e5920f3f0b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:03.635303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ff1f9f6d-a41a-41a5-966a-40e5920f3f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:03.700930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:03.707723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f9f6d-a41a-41a5-966a-40e5920f3f0b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:03.714178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:03.715208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:03.715499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:03.715776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] PATH_INFO: `/v3/users/5327cb42ab4a40c88493279f5bb85505` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:03.717054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5327cb42ab4a40c88493279f5bb85505)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:03.719022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:03.778634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5327cb42ab4a40c88493279f5bb85505'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:03.796965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5327cb42ab4a40c88493279f5bb85505'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:03.797801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5327cb42ab4a40c88493279f5bb85505'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:03.798490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5327cb42ab4a40c88493279f5bb85505'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:03.805599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5327cb42ab4a40c88493279f5bb85505'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:03.821967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1000/1987] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:03 2022] DELETE /identity/v3/users/5327cb42ab4a40c88493279f5bb85505 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:03.827917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b19b5-c12e-4caf-8f5a-3d758f95d9bf None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:03.828386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b19b5-c12e-4caf-8f5a-3d758f95d9bf None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:03.828472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b19b5-c12e-4caf-8f5a-3d758f95d9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:03.847343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-940b19b5-c12e-4caf-8f5a-3d758f95d9bf None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:03.848728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-940b19b5-c12e-4caf-8f5a-3d758f95d9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:03.912675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 988/1988] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:03.922765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:03.924174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:04.012513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b6ca3-f9dc-4465-ad7e-22e0aaa98283 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:04.016381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e3c1c-2bbb-4fb8-ac44-4e5252c95629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:04.017066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e3c1c-2bbb-4fb8-ac44-4e5252c95629 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:04.017349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e3c1c-2bbb-4fb8-ac44-4e5252c95629 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:04.017621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e3c1c-2bbb-4fb8-ac44-4e5252c95629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:04.027875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e3c1c-2bbb-4fb8-ac44-4e5252c95629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:04.029364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e3c1c-2bbb-4fb8-ac44-4e5252c95629 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:04.040763 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1001/1989] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:04.153673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b19b5-c12e-4caf-8f5a-3d758f95d9bf None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:04.156111 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-940b19b5-c12e-4caf-8f5a-3d758f95d9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:04.174069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b19b5-c12e-4caf-8f5a-3d758f95d9bf None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:04.184538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:04.185865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:04.186892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:04.187423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] PATH_INFO: `/v3/users/1500a7221a444ac6966c2c1d1335b902` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:04.189493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1500a7221a444ac6966c2c1d1335b902)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:04.192426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:04.273906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1500a7221a444ac6966c2c1d1335b902'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.292611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1500a7221a444ac6966c2c1d1335b902'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.304510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1500a7221a444ac6966c2c1d1335b902'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.305657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1500a7221a444ac6966c2c1d1335b902'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.319386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1500a7221a444ac6966c2c1d1335b902'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.332454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 989/1990] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:04 2022] DELETE /identity/v3/users/1500a7221a444ac6966c2c1d1335b902 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:04.337774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a719aac-964f-4a34-9186-0d77085f3d9e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:04.338329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a719aac-964f-4a34-9186-0d77085f3d9e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:04.338704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a719aac-964f-4a34-9186-0d77085f3d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:04.359033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7a719aac-964f-4a34-9186-0d77085f3d9e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:04.360404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7a719aac-964f-4a34-9186-0d77085f3d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:04.466699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1002/1991] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:04.477276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:04.479899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:04.505904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a719aac-964f-4a34-9186-0d77085f3d9e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:04.507949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7a719aac-964f-4a34-9186-0d77085f3d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:04.519177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:04.526354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a719aac-964f-4a34-9186-0d77085f3d9e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:04.536121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:04.537186 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:04.537513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:04.538133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] PATH_INFO: `/v3/projects/13d4a4bf29e6431cb6c1f9b91b34a3e2` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:04.539349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d4a4bf29e6431cb6c1f9b91b34a3e2)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:04.541915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:04.562542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:04.581987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a7465-e428-4989-bf3d-67f92fcfe27c admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:04.585858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b003d-6003-4c70-a962-7a953ffa2881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:04.586609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b003d-6003-4c70-a962-7a953ffa2881 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:04.587575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b003d-6003-4c70-a962-7a953ffa2881 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:04.587575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b003d-6003-4c70-a962-7a953ffa2881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:04.600533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d4a4bf29e6431cb6c1f9b91b34a3e2', 'request_id': 'req-b64c3ff7-af53-4e98-a9c0-970fca547bf4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.600816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b003d-6003-4c70-a962-7a953ffa2881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:04.602379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b003d-6003-4c70-a962-7a953ffa2881 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:04.611072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d4a4bf29e6431cb6c1f9b91b34a3e2', 'request_id': 'req-b64c3ff7-af53-4e98-a9c0-970fca547bf4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.615673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 990/1992] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:04.619671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d4a4bf29e6431cb6c1f9b91b34a3e2', 'request_id': 'req-b64c3ff7-af53-4e98-a9c0-970fca547bf4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.633191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] The token cache is being invalidate because project 13d4a4bf29e6431cb6c1f9b91b34a3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:04.633488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.634190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.637010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1003/1993] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:04 2022] DELETE /identity/v3/projects/13d4a4bf29e6431cb6c1f9b91b34a3e2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:04.640676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b003d-6003-4c70-a962-7a953ffa2881 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:04.661014 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b003d-6003-4c70-a962-7a953ffa2881 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:04.714323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:04.716099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:04.748483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:04.749477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:04.749738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:04.750016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] PATH_INFO: `/v3/users/63614e3fd48e4dfba532efe78e5c7797` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:04.751405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63614e3fd48e4dfba532efe78e5c7797)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:04.753968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:04.817139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c3ff7-af53-4e98-a9c0-970fca547bf4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:04.824201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:04.825336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:04.825683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:04.826037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] PATH_INFO: `/v3/users/1c910e382192434aa80382a67a0bf5ad` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:04.827519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c910e382192434aa80382a67a0bf5ad)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:04.827828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63614e3fd48e4dfba532efe78e5c7797'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.829831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:04.841977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63614e3fd48e4dfba532efe78e5c7797'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.852738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63614e3fd48e4dfba532efe78e5c7797'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.853455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63614e3fd48e4dfba532efe78e5c7797'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.861773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8a71992f-1a24-4245-8469-9271ed02d841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63614e3fd48e4dfba532efe78e5c7797'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.872690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 991/1994] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:04 2022] DELETE /identity/v3/users/63614e3fd48e4dfba532efe78e5c7797 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:04.907276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c910e382192434aa80382a67a0bf5ad'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.922633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c910e382192434aa80382a67a0bf5ad'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.923418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c910e382192434aa80382a67a0bf5ad'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.924069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c910e382192434aa80382a67a0bf5ad'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.931795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c910e382192434aa80382a67a0bf5ad'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:04.947664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1004/1995] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:04 2022] DELETE /identity/v3/users/1c910e382192434aa80382a67a0bf5ad => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:04.952946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78ee02-ed40-4f00-90a7-90686bf6094b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:04.953345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78ee02-ed40-4f00-90a7-90686bf6094b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:04.953744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78ee02-ed40-4f00-90a7-90686bf6094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:04.976194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-1d78ee02-ed40-4f00-90a7-90686bf6094b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:04.977687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1d78ee02-ed40-4f00-90a7-90686bf6094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.025394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:05.047005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.048702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.075455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 992/1996] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:05.091225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:05.093182 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78ee02-ed40-4f00-90a7-90686bf6094b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.097350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1d78ee02-ed40-4f00-90a7-90686bf6094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.142929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046146f-4e3d-4426-b7ed-4e92b460bf09 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.149519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.150825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.150825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.151066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] PATH_INFO: `/v3/projects/d169407e7e614960aadfbc8542b73cc6` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.152326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d169407e7e614960aadfbc8542b73cc6)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.154291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.191473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78ee02-ed40-4f00-90a7-90686bf6094b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.196189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bfb07-32d2-459f-8f69-e355681c2b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.197051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bfb07-32d2-459f-8f69-e355681c2b17 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.197339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bfb07-32d2-459f-8f69-e355681c2b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.197613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bfb07-32d2-459f-8f69-e355681c2b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.208454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bfb07-32d2-459f-8f69-e355681c2b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.210055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bfb07-32d2-459f-8f69-e355681c2b17 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.231421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd169407e7e614960aadfbc8542b73cc6', 'request_id': 'req-c9133ce9-329a-4ec9-9a51-367ca363d8a4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.233163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 993/1997] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:05.234117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bfb07-32d2-459f-8f69-e355681c2b17 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.236718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-af3bfb07-32d2-459f-8f69-e355681c2b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.249124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd169407e7e614960aadfbc8542b73cc6', 'request_id': 'req-c9133ce9-329a-4ec9-9a51-367ca363d8a4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.256836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd169407e7e614960aadfbc8542b73cc6', 'request_id': 'req-c9133ce9-329a-4ec9-9a51-367ca363d8a4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.257088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bfb07-32d2-459f-8f69-e355681c2b17 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.279749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] The token cache is being invalidate because project d169407e7e614960aadfbc8542b73cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:05.280174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.281406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c9133ce9-329a-4ec9-9a51-367ca363d8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.283268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1005/1998] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:05 2022] DELETE /identity/v3/projects/d169407e7e614960aadfbc8542b73cc6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:05.288445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681bbe1-757e-47ad-acc2-05a3e9bb3afb None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.288853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681bbe1-757e-47ad-acc2-05a3e9bb3afb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.289290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681bbe1-757e-47ad-acc2-05a3e9bb3afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.314654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-e681bbe1-757e-47ad-acc2-05a3e9bb3afb None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:05.316479 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e681bbe1-757e-47ad-acc2-05a3e9bb3afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.334258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.335616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.336025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.336403 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] PATH_INFO: `/v3/projects/eec7458011ba4c0b939ab09a9ee99aac` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.337900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec7458011ba4c0b939ab09a9ee99aac)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.340347 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.444298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec7458011ba4c0b939ab09a9ee99aac', 'request_id': 'req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.452716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec7458011ba4c0b939ab09a9ee99aac', 'request_id': 'req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.461184 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec7458011ba4c0b939ab09a9ee99aac', 'request_id': 'req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.461466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1006/1999] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:05.470126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec25de-1ca8-4fd4-aa78-02c51ccc56d1 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.470662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec25de-1ca8-4fd4-aa78-02c51ccc56d1 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.471124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec25de-1ca8-4fd4-aa78-02c51ccc56d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.473599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] The token cache is being invalidate because project eec7458011ba4c0b939ab09a9ee99aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:05.473978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.474767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ac93a4a6-5e8c-4235-8536-a0a1c12e5fc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.476833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 994/2000] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:05 2022] DELETE /identity/v3/projects/eec7458011ba4c0b939ab09a9ee99aac => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:05.489278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11b4ae-aae0-4e92-8417-eef8c4c63bcb None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.489701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11b4ae-aae0-4e92-8417-eef8c4c63bcb None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.489991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11b4ae-aae0-4e92-8417-eef8c4c63bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.493452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-86ec25de-1ca8-4fd4-aa78-02c51ccc56d1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:05.494772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-86ec25de-1ca8-4fd4-aa78-02c51ccc56d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.515293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-ba11b4ae-aae0-4e92-8417-eef8c4c63bcb None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:05.516554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ba11b4ae-aae0-4e92-8417-eef8c4c63bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.527883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:05.554047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:05.576122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1007/2001] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:05.582383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec25de-1ca8-4fd4-aa78-02c51ccc56d1 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.591965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec25de-1ca8-4fd4-aa78-02c51ccc56d1 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.592496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 995/2002] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:05.599930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c1698-0be9-4a8b-8655-0dcb955ced3a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.600224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c1698-0be9-4a8b-8655-0dcb955ced3a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.600504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c1698-0be9-4a8b-8655-0dcb955ced3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.603043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72adfa2-0dbe-46aa-883b-b1d8969bfa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.603659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72adfa2-0dbe-46aa-883b-b1d8969bfa39 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.603940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72adfa2-0dbe-46aa-883b-b1d8969bfa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.604211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72adfa2-0dbe-46aa-883b-b1d8969bfa39 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.606043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72adfa2-0dbe-46aa-883b-b1d8969bfa39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:05.606043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72adfa2-0dbe-46aa-883b-b1d8969bfa39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.607445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72adfa2-0dbe-46aa-883b-b1d8969bfa39 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.621822 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-6d9c1698-0be9-4a8b-8655-0dcb955ced3a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:05.622974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6d9c1698-0be9-4a8b-8655-0dcb955ced3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.623726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1008/2003] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:50:05.627903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72adfa2-0dbe-46aa-883b-b1d8969bfa39 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.635456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 996/2004] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:05.639892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c1698-0be9-4a8b-8655-0dcb955ced3a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.640816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72adfa2-0dbe-46aa-883b-b1d8969bfa39 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.648608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875083a-c158-412f-a624-785c4704fb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.649445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875083a-c158-412f-a624-785c4704fb86 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.649556 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c1698-0be9-4a8b-8655-0dcb955ced3a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.649859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875083a-c158-412f-a624-785c4704fb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.650222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875083a-c158-412f-a624-785c4704fb86 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.651116 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875083a-c158-412f-a624-785c4704fb86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.653444 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875083a-c158-412f-a624-785c4704fb86 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.655040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.655752 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.656036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.656311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.657081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:05.657450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.659122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.678450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 997/2005] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:50:05.682566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.684384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.688275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1009/2006] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:05 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:05.688275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875083a-c158-412f-a624-785c4704fb86 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.697664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901f9eb-92b6-4dbe-bada-e8552a3b8394 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.757387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.758275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.758530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875083a-c158-412f-a624-785c4704fb86 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.758631 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.758832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] PATH_INFO: `/v3/projects/ccbe8abfccda48bc9aa0d201e2ef2a15` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.760170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccbe8abfccda48bc9aa0d201e2ef2a15)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.763155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.764502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3442a-1ce2-4bf3-a91b-f8a538832e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.766119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3442a-1ce2-4bf3-a91b-f8a538832e84 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.766119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3442a-1ce2-4bf3-a91b-f8a538832e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.766119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3442a-1ce2-4bf3-a91b-f8a538832e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.766401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3442a-1ce2-4bf3-a91b-f8a538832e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.768458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3442a-1ce2-4bf3-a91b-f8a538832e84 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.803902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1010/2007] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:05 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:05.803902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3442a-1ce2-4bf3-a91b-f8a538832e84 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.824024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccbe8abfccda48bc9aa0d201e2ef2a15', 'request_id': 'req-20fcc445-e27f-44dc-b0c0-4f2507bcd283'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.831714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccbe8abfccda48bc9aa0d201e2ef2a15', 'request_id': 'req-20fcc445-e27f-44dc-b0c0-4f2507bcd283'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.845105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccbe8abfccda48bc9aa0d201e2ef2a15', 'request_id': 'req-20fcc445-e27f-44dc-b0c0-4f2507bcd283'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.857335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] The token cache is being invalidate because project ccbe8abfccda48bc9aa0d201e2ef2a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:05.857335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.858612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:05.859882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 998/2008] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:05 2022] DELETE /identity/v3/projects/ccbe8abfccda48bc9aa0d201e2ef2a15 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:05.860708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.863712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.876672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3442a-1ce2-4bf3-a91b-f8a538832e84 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.892340 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:05.935511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:05.941865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc445-e27f-44dc-b0c0-4f2507bcd283 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.947209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aff76b-80ce-4cd9-bcba-8b13064796d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.947921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aff76b-80ce-4cd9-bcba-8b13064796d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.948207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aff76b-80ce-4cd9-bcba-8b13064796d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.948611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aff76b-80ce-4cd9-bcba-8b13064796d9 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.949237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f43508-09f6-4713-9ca2-e1837e3ff540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.949369 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aff76b-80ce-4cd9-bcba-8b13064796d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.950163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f43508-09f6-4713-9ca2-e1837e3ff540 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.950616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f43508-09f6-4713-9ca2-e1837e3ff540 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.951029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f43508-09f6-4713-9ca2-e1837e3ff540 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.951354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aff76b-80ce-4cd9-bcba-8b13064796d9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.952518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f43508-09f6-4713-9ca2-e1837e3ff540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.955169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f43508-09f6-4713-9ca2-e1837e3ff540 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.975654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 999/2009] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:05 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:05.976840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aff76b-80ce-4cd9-bcba-8b13064796d9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.978258 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1011/2010] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:05 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:05.980021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f43508-09f6-4713-9ca2-e1837e3ff540 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:05.980333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f5aff76b-80ce-4cd9-bcba-8b13064796d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:05.989862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f43508-09f6-4713-9ca2-e1837e3ff540 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.992871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aff76b-80ce-4cd9-bcba-8b13064796d9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:05.994688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c57c2-437c-4fd2-be86-d9686d2e454a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.995930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c57c2-437c-4fd2-be86-d9686d2e454a admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.996200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c57c2-437c-4fd2-be86-d9686d2e454a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.996479 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c57c2-437c-4fd2-be86-d9686d2e454a admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:05.997250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c57c2-437c-4fd2-be86-d9686d2e454a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:05.998369 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:05.998887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c57c2-437c-4fd2-be86-d9686d2e454a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:05.999233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:05.999541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:05.999833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] PATH_INFO: `/v3/users/5a6053f66c5045e6990b62c73a7dcbd2` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.001079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a6053f66c5045e6990b62c73a7dcbd2)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.002983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.009350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1012/2011] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:06.010149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c57c2-437c-4fd2-be86-d9686d2e454a admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.012831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-303c57c2-437c-4fd2-be86-d9686d2e454a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:06.025327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c57c2-437c-4fd2-be86-d9686d2e454a admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.030904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7fc96-2686-4991-8c34-87012d038ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.031591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7fc96-2686-4991-8c34-87012d038ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.032036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7fc96-2686-4991-8c34-87012d038ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.032350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7fc96-2686-4991-8c34-87012d038ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.033127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7fc96-2686-4991-8c34-87012d038ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.034734 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7fc96-2686-4991-8c34-87012d038ec3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.046106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1013/2012] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:06.049823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7fc96-2686-4991-8c34-87012d038ec3 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.053256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a6053f66c5045e6990b62c73a7dcbd2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.060191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7fc96-2686-4991-8c34-87012d038ec3 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.064837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb6c88-45ca-46b6-98cc-e1fea45dc35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.065491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb6c88-45ca-46b6-98cc-e1fea45dc35f admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.065774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb6c88-45ca-46b6-98cc-e1fea45dc35f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.066051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb6c88-45ca-46b6-98cc-e1fea45dc35f admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.066388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a6053f66c5045e6990b62c73a7dcbd2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.066775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb6c88-45ca-46b6-98cc-e1fea45dc35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.068356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb6c88-45ca-46b6-98cc-e1fea45dc35f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.076838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a6053f66c5045e6990b62c73a7dcbd2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.077503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a6053f66c5045e6990b62c73a7dcbd2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.079046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1014/2013] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:06.080246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb6c88-45ca-46b6-98cc-e1fea45dc35f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.085906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a6053f66c5045e6990b62c73a7dcbd2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.091695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb6c88-45ca-46b6-98cc-e1fea45dc35f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.092272 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1000/2014] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:05 2022] DELETE /identity/v3/users/5a6053f66c5045e6990b62c73a7dcbd2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:06.093058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.094385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:06.159427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec7d42-f6e2-40f3-aa23-d2ed2f38b99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.160155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec7d42-f6e2-40f3-aa23-d2ed2f38b99e admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.160503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec7d42-f6e2-40f3-aa23-d2ed2f38b99e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.160806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec7d42-f6e2-40f3-aa23-d2ed2f38b99e admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.161690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec7d42-f6e2-40f3-aa23-d2ed2f38b99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.163712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec7d42-f6e2-40f3-aa23-d2ed2f38b99e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.173261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a76fc1-eaa5-4a80-8b43-bfbe0e5ca343 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.179542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1015/2015] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:06.179721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a791d7c-5783-4020-8151-6984461cb852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.180310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a791d7c-5783-4020-8151-6984461cb852 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.180775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a791d7c-5783-4020-8151-6984461cb852 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.180895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec7d42-f6e2-40f3-aa23-d2ed2f38b99e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.181060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a791d7c-5783-4020-8151-6984461cb852 admin admin] PATH_INFO: `/v3/projects/d22470f87b13491abc061f19f0c76773/users/14f578dd57a64b00992486154f4f740f/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.183005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-22ec7d42-f6e2-40f3-aa23-d2ed2f38b99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:06.196964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec7d42-f6e2-40f3-aa23-d2ed2f38b99e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.201195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a791d7c-5783-4020-8151-6984461cb852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22470f87b13491abc061f19f0c76773, user_id=14f578dd57a64b00992486154f4f740f, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.203709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00337a-7732-4525-a653-c8cee208fbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.204863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00337a-7732-4525-a653-c8cee208fbba admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.205207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00337a-7732-4525-a653-c8cee208fbba admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.205578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00337a-7732-4525-a653-c8cee208fbba admin admin] PATH_INFO: `/v3/projects/b45f691b51284dbbb7f1fd8d482af7e4/users/6e416196a4614ccc90c8f2411e3f0258/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.205698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a791d7c-5783-4020-8151-6984461cb852 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.223541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1001/2016] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:06 2022] PUT /identity/v3/projects/d22470f87b13491abc061f19f0c76773/users/14f578dd57a64b00992486154f4f740f/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:06.224761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00337a-7732-4525-a653-c8cee208fbba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b45f691b51284dbbb7f1fd8d482af7e4, user_id=6e416196a4614ccc90c8f2411e3f0258, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.227553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00337a-7732-4525-a653-c8cee208fbba admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.235674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cbe1e-6008-4ea6-857f-82646dc65ee8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.235674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cbe1e-6008-4ea6-857f-82646dc65ee8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.236355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cbe1e-6008-4ea6-857f-82646dc65ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.248525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1016/2017] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:06 2022] PUT /identity/v3/projects/b45f691b51284dbbb7f1fd8d482af7e4/users/6e416196a4614ccc90c8f2411e3f0258/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:06.251164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8b403-22f8-4841-9bbf-3a25a3eb059e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.251583 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8b403-22f8-4841-9bbf-3a25a3eb059e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.251817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8b403-22f8-4841-9bbf-3a25a3eb059e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.259657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:06.271542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-6d0cbe1e-6008-4ea6-857f-82646dc65ee8 None None] MFA Rules not processed for user `14f578dd57a64b00992486154f4f740f`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:06.273586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6d0cbe1e-6008-4ea6-857f-82646dc65ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:06.274416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-12e8b403-22f8-4841-9bbf-3a25a3eb059e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:06.277296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-12e8b403-22f8-4841-9bbf-3a25a3eb059e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:06.356214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1017/2018] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:06.360352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1002/2019] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:06.363212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8b403-22f8-4841-9bbf-3a25a3eb059e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.364752 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cbe1e-6008-4ea6-857f-82646dc65ee8 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.364896 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-12e8b403-22f8-4841-9bbf-3a25a3eb059e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:06.377042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cbe1e-6008-4ea6-857f-82646dc65ee8 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.382413 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c2e7b-b809-4b42-a0e5-82d3e49c128e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.382699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c2e7b-b809-4b42-a0e5-82d3e49c128e admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.382986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c2e7b-b809-4b42-a0e5-82d3e49c128e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.383265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c2e7b-b809-4b42-a0e5-82d3e49c128e admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.383966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c2e7b-b809-4b42-a0e5-82d3e49c128e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.385957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c2e7b-b809-4b42-a0e5-82d3e49c128e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.391236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:06.419253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1003/2020] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:06 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:06.420292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c2e7b-b809-4b42-a0e5-82d3e49c128e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.422503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-207c2e7b-b809-4b42-a0e5-82d3e49c128e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:06.440554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8b403-22f8-4841-9bbf-3a25a3eb059e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.490453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c2e7b-b809-4b42-a0e5-82d3e49c128e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.496366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.497931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.498216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.498498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] PATH_INFO: `/v3/projects/2bebe0a95fcf48d0969475b921e9d3c1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.499737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bebe0a95fcf48d0969475b921e9d3c1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.501687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.501821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefb78b-e016-4e17-81c4-ede740bfc435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.502598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefb78b-e016-4e17-81c4-ede740bfc435 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.503045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefb78b-e016-4e17-81c4-ede740bfc435 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.503348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefb78b-e016-4e17-81c4-ede740bfc435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.515135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefb78b-e016-4e17-81c4-ede740bfc435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.516796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefb78b-e016-4e17-81c4-ede740bfc435 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.527326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1018/2021] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:06.528495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefb78b-e016-4e17-81c4-ede740bfc435 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.543379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefb78b-e016-4e17-81c4-ede740bfc435 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.564569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bebe0a95fcf48d0969475b921e9d3c1', 'request_id': 'req-560809fc-f8d5-4cd6-acee-afaefb156261'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.580976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bebe0a95fcf48d0969475b921e9d3c1', 'request_id': 'req-560809fc-f8d5-4cd6-acee-afaefb156261'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.592952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bebe0a95fcf48d0969475b921e9d3c1', 'request_id': 'req-560809fc-f8d5-4cd6-acee-afaefb156261'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.608652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] The token cache is being invalidate because project 2bebe0a95fcf48d0969475b921e9d3c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:06.608956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.611592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.613559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1004/2022] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:06 2022] DELETE /identity/v3/projects/2bebe0a95fcf48d0969475b921e9d3c1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:06.617175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.618654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:06.630804 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:06.633401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42caeb-7d86-4f7d-bf72-0ef856597329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.634443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42caeb-7d86-4f7d-bf72-0ef856597329 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.634920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42caeb-7d86-4f7d-bf72-0ef856597329 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.635194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42caeb-7d86-4f7d-bf72-0ef856597329 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.637098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42caeb-7d86-4f7d-bf72-0ef856597329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.640850 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42caeb-7d86-4f7d-bf72-0ef856597329 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.670510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1019/2023] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:06 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:06.672015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42caeb-7d86-4f7d-bf72-0ef856597329 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.674931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-af42caeb-7d86-4f7d-bf72-0ef856597329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:06.694604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42caeb-7d86-4f7d-bf72-0ef856597329 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.703651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b05024-2664-4e65-9254-12ac6ffabe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.703651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b05024-2664-4e65-9254-12ac6ffabe2f admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.703651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b05024-2664-4e65-9254-12ac6ffabe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.704972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b05024-2664-4e65-9254-12ac6ffabe2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.704972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b05024-2664-4e65-9254-12ac6ffabe2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.706872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b05024-2664-4e65-9254-12ac6ffabe2f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.718677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1020/2024] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:50:06.720393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560809fc-f8d5-4cd6-acee-afaefb156261 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.723141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b05024-2664-4e65-9254-12ac6ffabe2f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.725568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.726455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.726736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.727016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] PATH_INFO: `/v3/users/8918f136484e46eba5ef77ed5abdccb1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.728222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8918f136484e46eba5ef77ed5abdccb1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.730310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.744725 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b05024-2664-4e65-9254-12ac6ffabe2f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.750697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd10f4b-6c30-4fb3-8b74-d80e320840b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.751587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd10f4b-6c30-4fb3-8b74-d80e320840b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.751866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd10f4b-6c30-4fb3-8b74-d80e320840b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.752303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd10f4b-6c30-4fb3-8b74-d80e320840b9 admin admin] PATH_INFO: `/v3/projects/ad0df8074a7446d8944c69372364d4a0/users/93f82619c6154f2c9590d4c6ef402010/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.774844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd10f4b-6c30-4fb3-8b74-d80e320840b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0df8074a7446d8944c69372364d4a0, user_id=93f82619c6154f2c9590d4c6ef402010, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.779087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd10f4b-6c30-4fb3-8b74-d80e320840b9 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:06.802310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1021/2025] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:06 2022] PUT /identity/v3/projects/ad0df8074a7446d8944c69372364d4a0/users/93f82619c6154f2c9590d4c6ef402010/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:06.806219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd10f4b-6c30-4fb3-8b74-d80e320840b9 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:06.842099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8918f136484e46eba5ef77ed5abdccb1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.859072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8918f136484e46eba5ef77ed5abdccb1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.863211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:06.871791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8918f136484e46eba5ef77ed5abdccb1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.872706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8918f136484e46eba5ef77ed5abdccb1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.883880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8918f136484e46eba5ef77ed5abdccb1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:06.896719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1005/2026] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:06 2022] DELETE /identity/v3/users/8918f136484e46eba5ef77ed5abdccb1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:06.900909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd10f4b-6c30-4fb3-8b74-d80e320840b9 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:06.983869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe2e96-a1f6-4547-9c89-5e9eca004ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:06.985628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe2e96-a1f6-4547-9c89-5e9eca004ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:06.985919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe2e96-a1f6-4547-9c89-5e9eca004ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:06.986250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe2e96-a1f6-4547-9c89-5e9eca004ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:06.987063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe2e96-a1f6-4547-9c89-5e9eca004ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:06.989545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe2e96-a1f6-4547-9c89-5e9eca004ca5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:07.016138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1022/2027] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:50:07.016695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:07.018338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.039638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561caa0-762a-4828-b53c-c49265383b17 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:07.049416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cabb0-2cc5-420e-b1bc-b86f6455a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:07.050621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cabb0-2cc5-420e-b1bc-b86f6455a2df admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.050909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cabb0-2cc5-420e-b1bc-b86f6455a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.051190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cabb0-2cc5-420e-b1bc-b86f6455a2df admin admin] PATH_INFO: `/v3/projects/ad0df8074a7446d8944c69372364d4a0/users/93f82619c6154f2c9590d4c6ef402010/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.054682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cabb0-2cc5-420e-b1bc-b86f6455a2df admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0df8074a7446d8944c69372364d4a0, user_id=93f82619c6154f2c9590d4c6ef402010, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:07.059280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cabb0-2cc5-420e-b1bc-b86f6455a2df admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:07.075229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1006/2028] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:07 2022] PUT /identity/v3/projects/ad0df8074a7446d8944c69372364d4a0/users/93f82619c6154f2c9590d4c6ef402010/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:07.088336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c64a8-3788-46ca-bdee-231036f1a314 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.088751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c64a8-3788-46ca-bdee-231036f1a314 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.092510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c64a8-3788-46ca-bdee-231036f1a314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.132785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-4e1c64a8-3788-46ca-bdee-231036f1a314 None None] MFA Rules not processed for user `93f82619c6154f2c9590d4c6ef402010`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:07.134677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4e1c64a8-3788-46ca-bdee-231036f1a314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.244627 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1023/2029] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:07.256135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cabb0-2cc5-420e-b1bc-b86f6455a2df admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:07.257992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1f2cabb0-2cc5-420e-b1bc-b86f6455a2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.281277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c64a8-3788-46ca-bdee-231036f1a314 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:07.282777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4e1c64a8-3788-46ca-bdee-231036f1a314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.353344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cabb0-2cc5-420e-b1bc-b86f6455a2df admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:07.358260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bc2ec-e50d-43c1-adc8-8516cc375f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:07.359201 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bc2ec-e50d-43c1-adc8-8516cc375f3b service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.359726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bc2ec-e50d-43c1-adc8-8516cc375f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.360193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262bc2ec-e50d-43c1-adc8-8516cc375f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.375065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bc2ec-e50d-43c1-adc8-8516cc375f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:07.377727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262bc2ec-e50d-43c1-adc8-8516cc375f3b service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:07.383312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c64a8-3788-46ca-bdee-231036f1a314 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:07.389830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:07.390883 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.391367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.392125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1007/2030] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:50:07.396837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] PATH_INFO: `/v3/projects/2e7f8df56c274c2d8092b7fc1d850ffb` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.398590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e7f8df56c274c2d8092b7fc1d850ffb)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:07.400642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:07.408139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:07.461330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7f8df56c274c2d8092b7fc1d850ffb', 'request_id': 'req-767a48e0-b568-4da6-af2e-efbbbbad7470'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.474141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7f8df56c274c2d8092b7fc1d850ffb', 'request_id': 'req-767a48e0-b568-4da6-af2e-efbbbbad7470'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.484355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bc2ec-e50d-43c1-adc8-8516cc375f3b service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:07.484697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7f8df56c274c2d8092b7fc1d850ffb', 'request_id': 'req-767a48e0-b568-4da6-af2e-efbbbbad7470'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.497098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262bc2ec-e50d-43c1-adc8-8516cc375f3b service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:07.511095 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] The token cache is being invalidate because project 2e7f8df56c274c2d8092b7fc1d850ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:07.511556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.512453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-767a48e0-b568-4da6-af2e-efbbbbad7470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.513163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:07.514920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1024/2031] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:07 2022] DELETE /identity/v3/projects/2e7f8df56c274c2d8092b7fc1d850ffb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:07.515795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b991fcc-cd14-4f71-9fad-e0e6433abfd9 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.516125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b991fcc-cd14-4f71-9fad-e0e6433abfd9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.516407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b991fcc-cd14-4f71-9fad-e0e6433abfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.548470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3b991fcc-cd14-4f71-9fad-e0e6433abfd9 None None] MFA Rules not processed for user `6e416196a4614ccc90c8f2411e3f0258`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:07.549857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3b991fcc-cd14-4f71-9fad-e0e6433abfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.568029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:07.569346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.569628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.569907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] PATH_INFO: `/v3/users/14f578dd57a64b00992486154f4f740f` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.571841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14f578dd57a64b00992486154f4f740f)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:07.574137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:07.629692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14f578dd57a64b00992486154f4f740f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.636413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1025/2032] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:07.636413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239413d-287b-4e3b-ac02-2516864128b3 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.636413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239413d-287b-4e3b-ac02-2516864128b3 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.636413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239413d-287b-4e3b-ac02-2516864128b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.655442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1239413d-287b-4e3b-ac02-2516864128b3 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:07.656865 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1239413d-287b-4e3b-ac02-2516864128b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.658480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14f578dd57a64b00992486154f4f740f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.666263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1026/2033] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:07.666980 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239413d-287b-4e3b-ac02-2516864128b3 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:07.669357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1239413d-287b-4e3b-ac02-2516864128b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.671865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14f578dd57a64b00992486154f4f740f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.672916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14f578dd57a64b00992486154f4f740f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.681948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f693ebab-ea8e-4719-aabb-8d873f8b6165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14f578dd57a64b00992486154f4f740f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:07.689553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1008/2034] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:07 2022] DELETE /identity/v3/users/14f578dd57a64b00992486154f4f740f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:07.691156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e20f-b540-4ba2-8426-994257c81652 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.691496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e20f-b540-4ba2-8426-994257c81652 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.691876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e20f-b540-4ba2-8426-994257c81652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.716284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-ebc0e20f-b540-4ba2-8426-994257c81652 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:07.717758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ebc0e20f-b540-4ba2-8426-994257c81652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.739914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239413d-287b-4e3b-ac02-2516864128b3 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:07.772323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:07.792718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1009/2035] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:07.794528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c506bae-595f-427c-a3b4-f96b47deb4d7 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.794963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c506bae-595f-427c-a3b4-f96b47deb4d7 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.795380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c506bae-595f-427c-a3b4-f96b47deb4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.800882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfdcad-7d5c-40aa-a130-39df253ff389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:07.801719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfdcad-7d5c-40aa-a130-39df253ff389 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.802002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfdcad-7d5c-40aa-a130-39df253ff389 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.802279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cfdcad-7d5c-40aa-a130-39df253ff389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.804144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-57cfdcad-7d5c-40aa-a130-39df253ff389 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.814514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-9c506bae-595f-427c-a3b4-f96b47deb4d7 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:07.815921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9c506bae-595f-427c-a3b4-f96b47deb4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.825444 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1010/2036] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:07.826301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c506bae-595f-427c-a3b4-f96b47deb4d7 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:07.836468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c506bae-595f-427c-a3b4-f96b47deb4d7 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:07.843491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:07.844344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.844662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.844942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.846990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:07.847303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:07.848939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:07.871805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1011/2037] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:50:07.871805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:07.874178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:07.884879 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfdcad-7d5c-40aa-a130-39df253ff389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:07.886376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cfdcad-7d5c-40aa-a130-39df253ff389 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:07.893291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:07.895457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1027/2038] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:07.896347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfdcad-7d5c-40aa-a130-39df253ff389 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:07.908151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cfdcad-7d5c-40aa-a130-39df253ff389 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:07.913284 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c48b69-4861-4c17-b1e9-45004032d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:07.913945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c48b69-4861-4c17-b1e9-45004032d030 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.914230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c48b69-4861-4c17-b1e9-45004032d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.914510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c48b69-4861-4c17-b1e9-45004032d030 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.915207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c48b69-4861-4c17-b1e9-45004032d030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:07.917774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c48b69-4861-4c17-b1e9-45004032d030 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:07.940776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ce51c-91b0-4434-8bfb-0ef17629e54a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:07.945281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939aa2f-3671-47d8-87c7-f9c75ec0f4c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:07.945994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939aa2f-3671-47d8-87c7-f9c75ec0f4c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:07.946280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939aa2f-3671-47d8-87c7-f9c75ec0f4c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:07.946600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939aa2f-3671-47d8-87c7-f9c75ec0f4c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:07.947175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1028/2039] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:07 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:07.951570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c48b69-4861-4c17-b1e9-45004032d030 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:07.962153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939aa2f-3671-47d8-87c7-f9c75ec0f4c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:07.965448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939aa2f-3671-47d8-87c7-f9c75ec0f4c5 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:07.980743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1012/2040] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:08.072334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c48b69-4861-4c17-b1e9-45004032d030 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:08.083960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac9260-1fbe-4e05-a74b-f33bdf684268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:08.085071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac9260-1fbe-4e05-a74b-f33bdf684268 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:08.085592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac9260-1fbe-4e05-a74b-f33bdf684268 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:08.086174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac9260-1fbe-4e05-a74b-f33bdf684268 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:08.087306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac9260-1fbe-4e05-a74b-f33bdf684268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:08.090495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac9260-1fbe-4e05-a74b-f33bdf684268 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:08.122673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1029/2041] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:07 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:08.128339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939aa2f-3671-47d8-87c7-f9c75ec0f4c5 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:08.143626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939aa2f-3671-47d8-87c7-f9c75ec0f4c5 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:08.153587 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db63671-e0ce-4608-b619-df2bdff68f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:08.157707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db63671-e0ce-4608-b619-df2bdff68f19 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:08.158075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db63671-e0ce-4608-b619-df2bdff68f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:08.158701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db63671-e0ce-4608-b619-df2bdff68f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:08.159611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db63671-e0ce-4608-b619-df2bdff68f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:08.161395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db63671-e0ce-4608-b619-df2bdff68f19 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:08.178036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1013/2042] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:50:08.184415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac9260-1fbe-4e05-a74b-f33bdf684268 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:08.199839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac9260-1fbe-4e05-a74b-f33bdf684268 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:08.204810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0589b-409e-4592-8d1f-44f6ba19faa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:08.205494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0589b-409e-4592-8d1f-44f6ba19faa8 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:08.206399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0589b-409e-4592-8d1f-44f6ba19faa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:08.206399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0589b-409e-4592-8d1f-44f6ba19faa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:08.207037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d0589b-409e-4592-8d1f-44f6ba19faa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:08.208698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d0589b-409e-4592-8d1f-44f6ba19faa8 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:08.223936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1030/2043] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:08.225470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db63671-e0ce-4608-b619-df2bdff68f19 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:08.238614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db63671-e0ce-4608-b619-df2bdff68f19 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:08.247485 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe11384-79e4-463f-be16-7f7eca30b01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:08.248551 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe11384-79e4-463f-be16-7f7eca30b01a admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:08.248954 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe11384-79e4-463f-be16-7f7eca30b01a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:08.249298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe11384-79e4-463f-be16-7f7eca30b01a admin admin] PATH_INFO: `/v3/projects/011a18b3c205437680b8065741b49064/users/5d5bbdc4b18943f2bb9981a8ddc1b971/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:08.277513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe11384-79e4-463f-be16-7f7eca30b01a admin admin] RBAC: Authorizing `identity:create_grant(project_id=011a18b3c205437680b8065741b49064, user_id=5d5bbdc4b18943f2bb9981a8ddc1b971, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:08.280995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe11384-79e4-463f-be16-7f7eca30b01a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:08.310294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1014/2044] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:08 2022] PUT /identity/v3/projects/011a18b3c205437680b8065741b49064/users/5d5bbdc4b18943f2bb9981a8ddc1b971/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:08.318400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0589b-409e-4592-8d1f-44f6ba19faa8 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:08.320799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-98d0589b-409e-4592-8d1f-44f6ba19faa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:08.322455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c670e-2fad-49ad-a565-b65ae5ff754f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:08.324995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c670e-2fad-49ad-a565-b65ae5ff754f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:08.326017 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c670e-2fad-49ad-a565-b65ae5ff754f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:08.350031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0589b-409e-4592-8d1f-44f6ba19faa8 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:08.411965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-613c670e-2fad-49ad-a565-b65ae5ff754f None None] MFA Rules not processed for user `5d5bbdc4b18943f2bb9981a8ddc1b971`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:08.413262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-613c670e-2fad-49ad-a565-b65ae5ff754f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:08.419994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:08.453421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:08.454327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:08.454504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:08.454943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] PATH_INFO: `/v3/projects/d22470f87b13491abc061f19f0c76773` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:08.457363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22470f87b13491abc061f19f0c76773)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:08.460685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:08.532605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1015/2045] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:08.544051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c670e-2fad-49ad-a565-b65ae5ff754f None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:08.545549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-613c670e-2fad-49ad-a565-b65ae5ff754f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:08.551617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22470f87b13491abc061f19f0c76773', 'request_id': 'req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.563969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22470f87b13491abc061f19f0c76773', 'request_id': 'req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.577728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22470f87b13491abc061f19f0c76773', 'request_id': 'req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.592410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] The token cache is being invalidate because project d22470f87b13491abc061f19f0c76773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:08.592744 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.593484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.595090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1031/2046] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:08 2022] DELETE /identity/v3/projects/d22470f87b13491abc061f19f0c76773 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:08.599775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:08.601269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:08.623185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c670e-2fad-49ad-a565-b65ae5ff754f None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:08.651784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:08.672977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344c524-a464-4a9f-9ea1-2b18cc4b12c8 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:08.677716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:08.678243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:08.678662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:08.678901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:08.679024 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:08.679211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] PATH_INFO: `/v3/users/93f82619c6154f2c9590d4c6ef402010` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:08.679323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:08.679596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:08.680542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f82619c6154f2c9590d4c6ef402010)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:08.681502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:08.682452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:08.735791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f82619c6154f2c9590d4c6ef402010'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.749322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f82619c6154f2c9590d4c6ef402010'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.750048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f82619c6154f2c9590d4c6ef402010'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.750692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f82619c6154f2c9590d4c6ef402010'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.751313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:08.753111 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:08.759449 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-08eab79d-39d0-44b8-bf87-86ed0fb49d3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f82619c6154f2c9590d4c6ef402010'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.762174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:08.767181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1016/2047] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:08.785860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1032/2048] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:08 2022] DELETE /identity/v3/users/93f82619c6154f2c9590d4c6ef402010 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:08.798574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:08.800345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:08.815960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b673bd-bfc8-4a40-818a-32a1ee5f465b service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:08.885076 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:08.886330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:08.886449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:08.886660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] PATH_INFO: `/v3/users/5d5bbdc4b18943f2bb9981a8ddc1b971` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:08.888842 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d5bbdc4b18943f2bb9981a8ddc1b971)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:08.891375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:08.969590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d5bbdc4b18943f2bb9981a8ddc1b971'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:08.991840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d5bbdc4b18943f2bb9981a8ddc1b971'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:09.003686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d5bbdc4b18943f2bb9981a8ddc1b971'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:09.004710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d5bbdc4b18943f2bb9981a8ddc1b971'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:09.013696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d5bbdc4b18943f2bb9981a8ddc1b971'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:09.023356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1017/2049] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:08 2022] DELETE /identity/v3/users/5d5bbdc4b18943f2bb9981a8ddc1b971 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:09.025146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0200d-02ca-4b73-8e73-a5193ac5cf38 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.025438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0200d-02ca-4b73-8e73-a5193ac5cf38 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.025717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0200d-02ca-4b73-8e73-a5193ac5cf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.043722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-37d0200d-02ca-4b73-8e73-a5193ac5cf38 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:09.044986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-37d0200d-02ca-4b73-8e73-a5193ac5cf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:09.111366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:09.112891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:09.133958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1033/2050] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:09.134926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:09.141011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0200d-02ca-4b73-8e73-a5193ac5cf38 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:09.142625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-37d0200d-02ca-4b73-8e73-a5193ac5cf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:09.143538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e81814-1b15-4462-abcf-94c8a733553f admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:09.151216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:09.152492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.153084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.153588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] PATH_INFO: `/v3/projects/ad0df8074a7446d8944c69372364d4a0` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.155220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0df8074a7446d8944c69372364d4a0)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:09.157549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:09.213445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0df8074a7446d8944c69372364d4a0', 'request_id': 'req-fa447f35-e926-40e5-b402-70627f856ee7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:09.219529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0200d-02ca-4b73-8e73-a5193ac5cf38 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:09.222228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0df8074a7446d8944c69372364d4a0', 'request_id': 'req-fa447f35-e926-40e5-b402-70627f856ee7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:09.231459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0df8074a7446d8944c69372364d4a0', 'request_id': 'req-fa447f35-e926-40e5-b402-70627f856ee7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:09.246080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] The token cache is being invalidate because project ad0df8074a7446d8944c69372364d4a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:09.246704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:09.247948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-fa447f35-e926-40e5-b402-70627f856ee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:09.250653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1018/2051] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:09 2022] DELETE /identity/v3/projects/ad0df8074a7446d8944c69372364d4a0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:09.255461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d230de-fe45-4400-991e-b729b314cdb1 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.255759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d230de-fe45-4400-991e-b729b314cdb1 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.256042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d230de-fe45-4400-991e-b729b314cdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.278394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-44d230de-fe45-4400-991e-b729b314cdb1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:09.279637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-44d230de-fe45-4400-991e-b729b314cdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:09.282780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fd93c-cbe0-45a4-8130-24b7f75e0bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:09.283510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fd93c-cbe0-45a4-8130-24b7f75e0bef service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.283866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fd93c-cbe0-45a4-8130-24b7f75e0bef service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.284147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fd93c-cbe0-45a4-8130-24b7f75e0bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.285940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a71fd93c-cbe0-45a4-8130-24b7f75e0bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:09.349564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1019/2052] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:09.359433 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fa5da-b520-449d-bf45-9ea189d6928b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.359527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fa5da-b520-449d-bf45-9ea189d6928b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.359792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265fa5da-b520-449d-bf45-9ea189d6928b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.374771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fd93c-cbe0-45a4-8130-24b7f75e0bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:09.376307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fd93c-cbe0-45a4-8130-24b7f75e0bef service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:09.384809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:09.386607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-265fa5da-b520-449d-bf45-9ea189d6928b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:09.388241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-265fa5da-b520-449d-bf45-9ea189d6928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:09.400645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1034/2053] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:09.400847 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1020/2054] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:09.402864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fd93c-cbe0-45a4-8130-24b7f75e0bef service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:09.420390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fd93c-cbe0-45a4-8130-24b7f75e0bef service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:09.425861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115c75c-2d0a-4fc5-9151-4b924f159df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:09.426535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115c75c-2d0a-4fc5-9151-4b924f159df4 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.426816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115c75c-2d0a-4fc5-9151-4b924f159df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.427093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115c75c-2d0a-4fc5-9151-4b924f159df4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.427826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115c75c-2d0a-4fc5-9151-4b924f159df4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:09.428132 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115c75c-2d0a-4fc5-9151-4b924f159df4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:09.429943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115c75c-2d0a-4fc5-9151-4b924f159df4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:09.451836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1035/2055] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:50:09.453476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fa5da-b520-449d-bf45-9ea189d6928b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:09.465748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265fa5da-b520-449d-bf45-9ea189d6928b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:09.471633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec16492-2325-4e51-87c4-f320a6dba16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:09.472695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec16492-2325-4e51-87c4-f320a6dba16a admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.473152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec16492-2325-4e51-87c4-f320a6dba16a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.473574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec16492-2325-4e51-87c4-f320a6dba16a admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.474676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec16492-2325-4e51-87c4-f320a6dba16a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:09.477374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec16492-2325-4e51-87c4-f320a6dba16a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:09.543936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364c1db-087d-48cd-bb93-d552e416ae47 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.544884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364c1db-087d-48cd-bb93-d552e416ae47 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.546046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364c1db-087d-48cd-bb93-d552e416ae47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.565162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1021/2056] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:09 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:09.566659 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec16492-2325-4e51-87c4-f320a6dba16a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:09.586141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7364c1db-087d-48cd-bb93-d552e416ae47 None None] MFA Rules not processed for user `6e416196a4614ccc90c8f2411e3f0258`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:09.589551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7364c1db-087d-48cd-bb93-d552e416ae47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:09.692175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1036/2057] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:09.704078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:09.708408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec16492-2325-4e51-87c4-f320a6dba16a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:09.716798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef11ec-b7e4-4720-bfde-a9ac0e569079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:09.717588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef11ec-b7e4-4720-bfde-a9ac0e569079 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.717952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef11ec-b7e4-4720-bfde-a9ac0e569079 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.718465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef11ec-b7e4-4720-bfde-a9ac0e569079 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.719740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef11ec-b7e4-4720-bfde-a9ac0e569079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:09.722094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef11ec-b7e4-4720-bfde-a9ac0e569079 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:09.752527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1022/2058] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:09 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:09.754727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364c1db-087d-48cd-bb93-d552e416ae47 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:09.766315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364c1db-087d-48cd-bb93-d552e416ae47 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:09.772393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a14bd-8071-4096-91e3-5b0d3b4c0ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:09.773226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a14bd-8071-4096-91e3-5b0d3b4c0ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.773580 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a14bd-8071-4096-91e3-5b0d3b4c0ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.773948 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a14bd-8071-4096-91e3-5b0d3b4c0ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.774762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a14bd-8071-4096-91e3-5b0d3b4c0ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:09.777245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a14bd-8071-4096-91e3-5b0d3b4c0ae3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:09.791799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1037/2059] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:09.796675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef11ec-b7e4-4720-bfde-a9ac0e569079 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:09.809426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef11ec-b7e4-4720-bfde-a9ac0e569079 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:09.816475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe68817-9cb4-4ffb-887b-b5e81851c9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:09.817220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe68817-9cb4-4ffb-887b-b5e81851c9e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.817504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe68817-9cb4-4ffb-887b-b5e81851c9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.817780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe68817-9cb4-4ffb-887b-b5e81851c9e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.818518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe68817-9cb4-4ffb-887b-b5e81851c9e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:09.820421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe68817-9cb4-4ffb-887b-b5e81851c9e3 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:09.838863 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1023/2060] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:09.843891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a14bd-8071-4096-91e3-5b0d3b4c0ae3 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:09.861431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a14bd-8071-4096-91e3-5b0d3b4c0ae3 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:09.874253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e68b6d-03fd-4796-a130-3024934a45bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:09.875269 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e68b6d-03fd-4796-a130-3024934a45bf admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.875761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e68b6d-03fd-4796-a130-3024934a45bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.876115 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e68b6d-03fd-4796-a130-3024934a45bf admin admin] PATH_INFO: `/v3/projects/0d6eca66f9dd41128e5416bee1e9fcdc/users/326fa51732fd4105a49a7a24eb9fb1d1/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.895672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e68b6d-03fd-4796-a130-3024934a45bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6eca66f9dd41128e5416bee1e9fcdc, user_id=326fa51732fd4105a49a7a24eb9fb1d1, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:09.904814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e68b6d-03fd-4796-a130-3024934a45bf admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:09.924653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1038/2061] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:09 2022] PUT /identity/v3/projects/0d6eca66f9dd41128e5416bee1e9fcdc/users/326fa51732fd4105a49a7a24eb9fb1d1/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:09.936088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e6e5b-8b03-463d-9253-c347e236e55e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:09.936691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e6e5b-8b03-463d-9253-c347e236e55e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:09.936808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e6e5b-8b03-463d-9253-c347e236e55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:09.970799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-9c4e6e5b-8b03-463d-9253-c347e236e55e None None] MFA Rules not processed for user `326fa51732fd4105a49a7a24eb9fb1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:09.972226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9c4e6e5b-8b03-463d-9253-c347e236e55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:10.010214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e68b6d-03fd-4796-a130-3024934a45bf admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.013203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-08e68b6d-03fd-4796-a130-3024934a45bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:10.088735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1024/2062] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:10.092395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e6e5b-8b03-463d-9253-c347e236e55e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.114610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e68b6d-03fd-4796-a130-3024934a45bf admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.121599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.123033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.123466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.123888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] PATH_INFO: `/v3/projects/011a18b3c205437680b8065741b49064` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.125320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=011a18b3c205437680b8065741b49064)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.127390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:10.181506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e6e5b-8b03-463d-9253-c347e236e55e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.190346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787328b7-375c-4477-b42f-31c0f43b9e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.191124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787328b7-375c-4477-b42f-31c0f43b9e4a admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.191504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787328b7-375c-4477-b42f-31c0f43b9e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.191959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787328b7-375c-4477-b42f-31c0f43b9e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.193131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787328b7-375c-4477-b42f-31c0f43b9e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.195281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787328b7-375c-4477-b42f-31c0f43b9e4a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:10.205937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011a18b3c205437680b8065741b49064', 'request_id': 'req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:10.222817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011a18b3c205437680b8065741b49064', 'request_id': 'req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:10.226062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:10.237805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1025/2063] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:10 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:10.241722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787328b7-375c-4477-b42f-31c0f43b9e4a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.242558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011a18b3c205437680b8065741b49064', 'request_id': 'req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:10.269999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] The token cache is being invalidate because project 011a18b3c205437680b8065741b49064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:10.270191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:10.274153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd6fab1b-a1b2-49e0-a9fe-62208bcb2b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:10.276834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1039/2064] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:10 2022] DELETE /identity/v3/projects/011a18b3c205437680b8065741b49064 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:10.284100 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa712f-89cd-4504-9514-7b691a4e8b9c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.284388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa712f-89cd-4504-9514-7b691a4e8b9c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.284703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa712f-89cd-4504-9514-7b691a4e8b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.318411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-64fa712f-89cd-4504-9514-7b691a4e8b9c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:10.320661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-64fa712f-89cd-4504-9514-7b691a4e8b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:10.321917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:10.357598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787328b7-375c-4477-b42f-31c0f43b9e4a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.372985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fa53f-1dc7-40aa-b6a8-6a4096a18fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.374734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fa53f-1dc7-40aa-b6a8-6a4096a18fba admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.375027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fa53f-1dc7-40aa-b6a8-6a4096a18fba admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.375310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fa53f-1dc7-40aa-b6a8-6a4096a18fba admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.376031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fa53f-1dc7-40aa-b6a8-6a4096a18fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.378051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fa53f-1dc7-40aa-b6a8-6a4096a18fba admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:10.418093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1040/2065] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:10.426018 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1be49-4beb-49c9-83fd-e359d723d631 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.426594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1be49-4beb-49c9-83fd-e359d723d631 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.427425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1be49-4beb-49c9-83fd-e359d723d631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.430762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1026/2066] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:10 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:10.431706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fa53f-1dc7-40aa-b6a8-6a4096a18fba admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.435548 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-bf7fa53f-1dc7-40aa-b6a8-6a4096a18fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:10.453774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fa53f-1dc7-40aa-b6a8-6a4096a18fba admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.462701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44168b5d-4da6-4b5a-bafb-28fad35e622e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.464749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44168b5d-4da6-4b5a-bafb-28fad35e622e admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.466491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44168b5d-4da6-4b5a-bafb-28fad35e622e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.466984 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44168b5d-4da6-4b5a-bafb-28fad35e622e admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.468066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44168b5d-4da6-4b5a-bafb-28fad35e622e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.468563 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-86e1be49-4beb-49c9-83fd-e359d723d631 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:10.470650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44168b5d-4da6-4b5a-bafb-28fad35e622e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:10.475360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-86e1be49-4beb-49c9-83fd-e359d723d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:10.491043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1041/2067] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:10.495687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1be49-4beb-49c9-83fd-e359d723d631 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.496541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1027/2068] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:10.497068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44168b5d-4da6-4b5a-bafb-28fad35e622e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.518605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1be49-4beb-49c9-83fd-e359d723d631 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.523158 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44168b5d-4da6-4b5a-bafb-28fad35e622e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.535278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf71b4-0353-4a76-b24e-aedc8919cb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.536294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf71b4-0353-4a76-b24e-aedc8919cb95 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.536756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf71b4-0353-4a76-b24e-aedc8919cb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.537183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf71b4-0353-4a76-b24e-aedc8919cb95 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.538927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf71b4-0353-4a76-b24e-aedc8919cb95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.543326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b6eee-307f-4198-92dd-0540debd5d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.544314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b6eee-307f-4198-92dd-0540debd5d64 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.544827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b6eee-307f-4198-92dd-0540debd5d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.545255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b6eee-307f-4198-92dd-0540debd5d64 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.546390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b6eee-307f-4198-92dd-0540debd5d64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:10.546859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b6eee-307f-4198-92dd-0540debd5d64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.549448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b6eee-307f-4198-92dd-0540debd5d64 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:10.552061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf71b4-0353-4a76-b24e-aedc8919cb95 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:10.579737 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1042/2069] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:10.580688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf71b4-0353-4a76-b24e-aedc8919cb95 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.594396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1028/2070] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:50:10.598913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b6eee-307f-4198-92dd-0540debd5d64 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.606256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf71b4-0353-4a76-b24e-aedc8919cb95 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.615415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b6eee-307f-4198-92dd-0540debd5d64 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.619634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e015f8-9d99-4ffd-8f2d-b817279f2e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.621363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e015f8-9d99-4ffd-8f2d-b817279f2e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.621645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e015f8-9d99-4ffd-8f2d-b817279f2e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.621919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e015f8-9d99-4ffd-8f2d-b817279f2e2e admin admin] PATH_INFO: `/v3/projects/a4c16ec3efe947d9b329abafaa030c08/users/64537005577044d788028d7dcc74e6c8/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.628083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cd1c7-1f40-4b1f-9797-1c17081abbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.629664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cd1c7-1f40-4b1f-9797-1c17081abbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.630163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cd1c7-1f40-4b1f-9797-1c17081abbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.630652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cd1c7-1f40-4b1f-9797-1c17081abbf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.632484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cd1c7-1f40-4b1f-9797-1c17081abbf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.636065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cd1c7-1f40-4b1f-9797-1c17081abbf8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:10.660667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e015f8-9d99-4ffd-8f2d-b817279f2e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c16ec3efe947d9b329abafaa030c08, user_id=64537005577044d788028d7dcc74e6c8, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.664226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e015f8-9d99-4ffd-8f2d-b817279f2e2e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:10.690726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1029/2071] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:10 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:10.691996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cd1c7-1f40-4b1f-9797-1c17081abbf8 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.711867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1043/2072] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:10 2022] PUT /identity/v3/projects/a4c16ec3efe947d9b329abafaa030c08/users/64537005577044d788028d7dcc74e6c8/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:10.714278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2b148-8552-4b78-a51b-9218bec3f6c4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.715036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2b148-8552-4b78-a51b-9218bec3f6c4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.715655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2b148-8552-4b78-a51b-9218bec3f6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.772390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b8d2b148-8552-4b78-a51b-9218bec3f6c4 None None] MFA Rules not processed for user `64537005577044d788028d7dcc74e6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:10.774267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b8d2b148-8552-4b78-a51b-9218bec3f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:10.797131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cd1c7-1f40-4b1f-9797-1c17081abbf8 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.861038 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1044/2073] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:10.862943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2b148-8552-4b78-a51b-9218bec3f6c4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.889465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3806190-9e5b-44a4-82c2-e77fd3b26ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.890154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3806190-9e5b-44a4-82c2-e77fd3b26ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.890427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3806190-9e5b-44a4-82c2-e77fd3b26ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.890746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3806190-9e5b-44a4-82c2-e77fd3b26ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.891622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3806190-9e5b-44a4-82c2-e77fd3b26ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.893682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3806190-9e5b-44a4-82c2-e77fd3b26ee9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:10.926868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1030/2074] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:10 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:10.932099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3806190-9e5b-44a4-82c2-e77fd3b26ee9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:10.950552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3806190-9e5b-44a4-82c2-e77fd3b26ee9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:10.967301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:10.973475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2b9da-2b0f-4cf5-9a2d-116299289fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:10.974253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2b9da-2b0f-4cf5-9a2d-116299289fab admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:10.974572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2b9da-2b0f-4cf5-9a2d-116299289fab admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:10.974850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2b9da-2b0f-4cf5-9a2d-116299289fab admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:10.975607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2b9da-2b0f-4cf5-9a2d-116299289fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:10.980386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d2b9da-2b0f-4cf5-9a2d-116299289fab admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.016806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1031/2075] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:11.017775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2b9da-2b0f-4cf5-9a2d-116299289fab admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:11.038990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2b9da-2b0f-4cf5-9a2d-116299289fab admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:11.046074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2b148-8552-4b78-a51b-9218bec3f6c4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:11.053536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e46f0-21d1-4312-9af7-1b830d6cce9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:11.054794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e46f0-21d1-4312-9af7-1b830d6cce9d admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.055851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e46f0-21d1-4312-9af7-1b830d6cce9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.055851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e46f0-21d1-4312-9af7-1b830d6cce9d admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.056267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e46f0-21d1-4312-9af7-1b830d6cce9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:11.059663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e46f0-21d1-4312-9af7-1b830d6cce9d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.061899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f4591-254d-415e-98c1-0397435896a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:11.075765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f4591-254d-415e-98c1-0397435896a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.076864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f4591-254d-415e-98c1-0397435896a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.077958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f4591-254d-415e-98c1-0397435896a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.081685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f4591-254d-415e-98c1-0397435896a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:11.086010 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1032/2076] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:11.094357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f4591-254d-415e-98c1-0397435896a5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.096426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e46f0-21d1-4312-9af7-1b830d6cce9d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:11.111425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e46f0-21d1-4312-9af7-1b830d6cce9d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:11.122002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7fa2a-fa7b-4b50-8df7-31cc982e735b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:11.124442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7fa2a-fa7b-4b50-8df7-31cc982e735b admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.124791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7fa2a-fa7b-4b50-8df7-31cc982e735b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.125107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7fa2a-fa7b-4b50-8df7-31cc982e735b admin admin] PATH_INFO: `/v3/projects/07b87adb66004fb39c8558287d1bee98/users/ac07e1b2de6f4607a37047b6edee4704/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.129097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:11.138469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1045/2077] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:10 2022] POST /identity/v3/projects => generated 385 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:11.141553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f4591-254d-415e-98c1-0397435896a5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:11.147529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7fa2a-fa7b-4b50-8df7-31cc982e735b admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b87adb66004fb39c8558287d1bee98, user_id=ac07e1b2de6f4607a37047b6edee4704, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:11.153609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7fa2a-fa7b-4b50-8df7-31cc982e735b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.171802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1033/2078] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:11 2022] PUT /identity/v3/projects/07b87adb66004fb39c8558287d1bee98/users/ac07e1b2de6f4607a37047b6edee4704/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:11.182924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935828b-6eec-4746-9ee3-f29c3acb2b8d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.183773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935828b-6eec-4746-9ee3-f29c3acb2b8d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.184306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935828b-6eec-4746-9ee3-f29c3acb2b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.230051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-6935828b-6eec-4746-9ee3-f29c3acb2b8d None None] MFA Rules not processed for user `ac07e1b2de6f4607a37047b6edee4704`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:11.238891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f4591-254d-415e-98c1-0397435896a5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:11.241949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6935828b-6eec-4746-9ee3-f29c3acb2b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:11.325760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e12b4-49df-4eaa-9815-93eb663f3ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:11.326731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e12b4-49df-4eaa-9815-93eb663f3ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.327014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e12b4-49df-4eaa-9815-93eb663f3ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.327287 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e12b4-49df-4eaa-9815-93eb663f3ac6 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.328878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e12b4-49df-4eaa-9815-93eb663f3ac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:11.331752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e12b4-49df-4eaa-9815-93eb663f3ac6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.350709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1034/2079] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:11.364526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1046/2080] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:11 2022] POST /identity/v3/users => generated 299 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:11.368043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935828b-6eec-4746-9ee3-f29c3acb2b8d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:11.369881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6935828b-6eec-4746-9ee3-f29c3acb2b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:11.370191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e12b4-49df-4eaa-9815-93eb663f3ac6 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:11.382490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e12b4-49df-4eaa-9815-93eb663f3ac6 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:11.391491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87decd36-9635-4193-856c-593413fa60e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:11.391913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87decd36-9635-4193-856c-593413fa60e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.392278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87decd36-9635-4193-856c-593413fa60e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.394927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87decd36-9635-4193-856c-593413fa60e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.395040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87decd36-9635-4193-856c-593413fa60e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:11.399631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87decd36-9635-4193-856c-593413fa60e7 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.417946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1047/2081] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:11.423834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87decd36-9635-4193-856c-593413fa60e7 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:11.439644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87decd36-9635-4193-856c-593413fa60e7 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:11.455332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464da6b-e03b-41f1-8e27-f97e83a0cb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:11.455607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464da6b-e03b-41f1-8e27-f97e83a0cb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.455891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464da6b-e03b-41f1-8e27-f97e83a0cb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.456165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464da6b-e03b-41f1-8e27-f97e83a0cb96 admin admin] PATH_INFO: `/v3/projects/72957804b3464427bda227e4f2d98d9d/users/60c82864b86f4a8faa4d850dd57b9064/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.480888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464da6b-e03b-41f1-8e27-f97e83a0cb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72957804b3464427bda227e4f2d98d9d, user_id=60c82864b86f4a8faa4d850dd57b9064, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:11.485309 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464da6b-e03b-41f1-8e27-f97e83a0cb96 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.487501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935828b-6eec-4746-9ee3-f29c3acb2b8d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:11.502370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1048/2082] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:11 2022] PUT /identity/v3/projects/72957804b3464427bda227e4f2d98d9d/users/60c82864b86f4a8faa4d850dd57b9064/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:11.506473 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464da6b-e03b-41f1-8e27-f97e83a0cb96 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:11.507281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204191e-9fda-461a-bfdb-76cdfe08fdd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:11.507969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204191e-9fda-461a-bfdb-76cdfe08fdd8 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.508252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204191e-9fda-461a-bfdb-76cdfe08fdd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.508533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204191e-9fda-461a-bfdb-76cdfe08fdd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.528561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204191e-9fda-461a-bfdb-76cdfe08fdd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:11.530422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204191e-9fda-461a-bfdb-76cdfe08fdd8 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.532118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:11.561332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1035/2083] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:11.614121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464da6b-e03b-41f1-8e27-f97e83a0cb96 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:11.620974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013e475-f556-4ac8-8a2e-03ee40646af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:11.622224 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e475-f556-4ac8-8a2e-03ee40646af3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.623213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e475-f556-4ac8-8a2e-03ee40646af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.623702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e475-f556-4ac8-8a2e-03ee40646af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.624808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013e475-f556-4ac8-8a2e-03ee40646af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:11.626827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013e475-f556-4ac8-8a2e-03ee40646af3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.640705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1049/2084] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:50:11.644418 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204191e-9fda-461a-bfdb-76cdfe08fdd8 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:11.663069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204191e-9fda-461a-bfdb-76cdfe08fdd8 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:11.670422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc1468-aaca-4254-a8b5-d368cb2e24ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:11.671461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc1468-aaca-4254-a8b5-d368cb2e24ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.671738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc1468-aaca-4254-a8b5-d368cb2e24ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.672013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc1468-aaca-4254-a8b5-d368cb2e24ea admin admin] PATH_INFO: `/v3/projects/72957804b3464427bda227e4f2d98d9d/users/60c82864b86f4a8faa4d850dd57b9064/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.674111 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc1468-aaca-4254-a8b5-d368cb2e24ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=72957804b3464427bda227e4f2d98d9d, user_id=60c82864b86f4a8faa4d850dd57b9064, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:11.676848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc1468-aaca-4254-a8b5-d368cb2e24ea admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:11.689278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1036/2085] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:11 2022] PUT /identity/v3/projects/72957804b3464427bda227e4f2d98d9d/users/60c82864b86f4a8faa4d850dd57b9064/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:11.701568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e67cf-d695-4a04-866c-c2e269b04185 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:11.703055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e67cf-d695-4a04-866c-c2e269b04185 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:11.703413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e67cf-d695-4a04-866c-c2e269b04185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:11.758054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f29e67cf-d695-4a04-866c-c2e269b04185 None None] MFA Rules not processed for user `60c82864b86f4a8faa4d850dd57b9064`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:11.759480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f29e67cf-d695-4a04-866c-c2e269b04185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:11.854089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1050/2086] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:11.865699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc1468-aaca-4254-a8b5-d368cb2e24ea admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:11.868880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a0cc1468-aaca-4254-a8b5-d368cb2e24ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:12.077377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc1468-aaca-4254-a8b5-d368cb2e24ea admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:12.094934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6c891-f2ed-4ac0-bb47-057fcafbed6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:12.095943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6c891-f2ed-4ac0-bb47-057fcafbed6c service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:12.096930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6c891-f2ed-4ac0-bb47-057fcafbed6c service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:12.097282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6c891-f2ed-4ac0-bb47-057fcafbed6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:12.128105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6c891-f2ed-4ac0-bb47-057fcafbed6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:12.130489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6c891-f2ed-4ac0-bb47-057fcafbed6c service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:12.154297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1037/2087] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:12.989260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e67cf-d695-4a04-866c-c2e269b04185 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:13.001233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e67cf-d695-4a04-866c-c2e269b04185 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:13.006796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a8343-7b3f-471b-826b-f45563ef9e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:13.007479 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a8343-7b3f-471b-826b-f45563ef9e84 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:13.007769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a8343-7b3f-471b-826b-f45563ef9e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:13.008053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a8343-7b3f-471b-826b-f45563ef9e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:13.009983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0f1a8343-7b3f-471b-826b-f45563ef9e84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:13.090589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a8343-7b3f-471b-826b-f45563ef9e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:13.092905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a8343-7b3f-471b-826b-f45563ef9e84 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:13.103910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1051/2088] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:13.140455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6c891-f2ed-4ac0-bb47-057fcafbed6c service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:13.142299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-70c6c891-f2ed-4ac0-bb47-057fcafbed6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:13.151174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ccaf0-b162-4ebb-870b-f2ef3b6b8bd4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:13.151889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ccaf0-b162-4ebb-870b-f2ef3b6b8bd4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:13.152117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ccaf0-b162-4ebb-870b-f2ef3b6b8bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:13.171567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:13.178732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-596ccaf0-b162-4ebb-870b-f2ef3b6b8bd4 None None] MFA Rules not processed for user `ac07e1b2de6f4607a37047b6edee4704`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:13.179983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-596ccaf0-b162-4ebb-870b-f2ef3b6b8bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:13.231766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:13.256624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1052/2089] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:13.259960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6c891-f2ed-4ac0-bb47-057fcafbed6c service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:13.265832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:13.266715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:13.267036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:13.267257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] PATH_INFO: `/v3/users/151db87b1e7248d882108519fab1d938` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:13.268456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=151db87b1e7248d882108519fab1d938)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:13.270444 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:13.333422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151db87b1e7248d882108519fab1d938'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:13.357834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151db87b1e7248d882108519fab1d938'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:13.369511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151db87b1e7248d882108519fab1d938'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:13.370338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151db87b1e7248d882108519fab1d938'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:13.384482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c263597-edb8-4603-9529-62cd07dddc49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151db87b1e7248d882108519fab1d938'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:13.395130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1038/2090] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:13 2022] DELETE /identity/v3/users/151db87b1e7248d882108519fab1d938 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:13.934447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ccaf0-b162-4ebb-870b-f2ef3b6b8bd4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:13.946107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ccaf0-b162-4ebb-870b-f2ef3b6b8bd4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:14.066039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:14.067134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.067497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.067919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] PATH_INFO: `/v3/projects/4921002ea57c4f54a7a3b69d802a6430` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.069264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4921002ea57c4f54a7a3b69d802a6430)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:14.072131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:14.135229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4921002ea57c4f54a7a3b69d802a6430', 'request_id': 'req-a78b9a04-dbe4-459f-a7be-786340c952aa'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:14.147933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4921002ea57c4f54a7a3b69d802a6430', 'request_id': 'req-a78b9a04-dbe4-459f-a7be-786340c952aa'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:14.156550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4921002ea57c4f54a7a3b69d802a6430', 'request_id': 'req-a78b9a04-dbe4-459f-a7be-786340c952aa'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:14.167310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] The token cache is being invalidate because project 4921002ea57c4f54a7a3b69d802a6430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:14.167603 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:14.168280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a78b9a04-dbe4-459f-a7be-786340c952aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:14.169887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1053/2091] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:13 2022] DELETE /identity/v3/projects/4921002ea57c4f54a7a3b69d802a6430 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:14.175012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac283-d5b2-4229-8244-cfcc827c7227 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.175513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac283-d5b2-4229-8244-cfcc827c7227 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.175608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac283-d5b2-4229-8244-cfcc827c7227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.201791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-64eac283-d5b2-4229-8244-cfcc827c7227 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:14.203374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-64eac283-d5b2-4229-8244-cfcc827c7227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:14.304411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1039/2092] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:14.313044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4ff93-e467-4f16-b9bd-63d5931b819f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.313305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4ff93-e467-4f16-b9bd-63d5931b819f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.313581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4ff93-e467-4f16-b9bd-63d5931b819f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.333364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d3c4ff93-e467-4f16-b9bd-63d5931b819f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:14.334604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d3c4ff93-e467-4f16-b9bd-63d5931b819f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:14.353158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1054/2093] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:14.358125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eac283-d5b2-4229-8244-cfcc827c7227 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:14.368727 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eac283-d5b2-4229-8244-cfcc827c7227 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:14.376499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860f65d-9a7f-4379-9e85-1c34c3398eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:14.377455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860f65d-9a7f-4379-9e85-1c34c3398eec admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.377819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860f65d-9a7f-4379-9e85-1c34c3398eec admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.378174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860f65d-9a7f-4379-9e85-1c34c3398eec admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.379016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860f65d-9a7f-4379-9e85-1c34c3398eec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:14.379407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860f65d-9a7f-4379-9e85-1c34c3398eec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:14.381197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860f65d-9a7f-4379-9e85-1c34c3398eec admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:14.393637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1040/2094] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:50:14.398343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4ff93-e467-4f16-b9bd-63d5931b819f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:14.408403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4ff93-e467-4f16-b9bd-63d5931b819f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:14.413536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2600-ebe9-48f5-a60c-5a18df8bf419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:14.414286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2600-ebe9-48f5-a60c-5a18df8bf419 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.414650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2600-ebe9-48f5-a60c-5a18df8bf419 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.415003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2600-ebe9-48f5-a60c-5a18df8bf419 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.415785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2600-ebe9-48f5-a60c-5a18df8bf419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:14.418469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2600-ebe9-48f5-a60c-5a18df8bf419 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:14.452279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1055/2095] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:14 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:14.455779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860f65d-9a7f-4379-9e85-1c34c3398eec admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:14.466355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860f65d-9a7f-4379-9e85-1c34c3398eec admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:14.485368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:14.539632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef051a98-318e-4203-9bfb-4df486ea3ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:14.540464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef051a98-318e-4203-9bfb-4df486ea3ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.540782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef051a98-318e-4203-9bfb-4df486ea3ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.541054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef051a98-318e-4203-9bfb-4df486ea3ea5 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.542545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef051a98-318e-4203-9bfb-4df486ea3ea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:14.544493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef051a98-318e-4203-9bfb-4df486ea3ea5 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:14.566449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1041/2096] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:14 2022] POST /identity/v3/users => generated 282 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:14.570437 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2600-ebe9-48f5-a60c-5a18df8bf419 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:14.586293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2600-ebe9-48f5-a60c-5a18df8bf419 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:14.592030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac59c87-76e6-4247-afb9-67f21c42b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:14.592849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac59c87-76e6-4247-afb9-67f21c42b09c admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.593211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac59c87-76e6-4247-afb9-67f21c42b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.593573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac59c87-76e6-4247-afb9-67f21c42b09c admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.594441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac59c87-76e6-4247-afb9-67f21c42b09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:14.596203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac59c87-76e6-4247-afb9-67f21c42b09c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:14.616550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1056/2097] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:14.621394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef051a98-318e-4203-9bfb-4df486ea3ea5 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:14.640626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef051a98-318e-4203-9bfb-4df486ea3ea5 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:14.640626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05ef59-4fb0-4363-8aa6-7d164a39ab33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:14.640626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05ef59-4fb0-4363-8aa6-7d164a39ab33 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.640626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05ef59-4fb0-4363-8aa6-7d164a39ab33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.640955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05ef59-4fb0-4363-8aa6-7d164a39ab33 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.640955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05ef59-4fb0-4363-8aa6-7d164a39ab33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:14.642673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05ef59-4fb0-4363-8aa6-7d164a39ab33 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:14.653365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1042/2098] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:50:14.658065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac59c87-76e6-4247-afb9-67f21c42b09c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:14.669013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac59c87-76e6-4247-afb9-67f21c42b09c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:14.674319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110057f1-e853-48f4-a5e0-69daea6a9c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:14.675219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110057f1-e853-48f4-a5e0-69daea6a9c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.675501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110057f1-e853-48f4-a5e0-69daea6a9c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.675778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110057f1-e853-48f4-a5e0-69daea6a9c3c admin admin] PATH_INFO: `/v3/projects/3bbadf83a57c496495289ad4cc93f135/users/f9a22f4b631f4fbf9b3d7795211eeccb/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.691964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110057f1-e853-48f4-a5e0-69daea6a9c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bbadf83a57c496495289ad4cc93f135, user_id=f9a22f4b631f4fbf9b3d7795211eeccb, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:14.694617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110057f1-e853-48f4-a5e0-69daea6a9c3c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:14.709580 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1057/2099] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:14 2022] PUT /identity/v3/projects/3bbadf83a57c496495289ad4cc93f135/users/f9a22f4b631f4fbf9b3d7795211eeccb/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:14.720681 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb73bc7-b4ac-4720-86a4-f9b7092b9f13 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.720977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb73bc7-b4ac-4720-86a4-f9b7092b9f13 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.721288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb73bc7-b4ac-4720-86a4-f9b7092b9f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.753516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-7cb73bc7-b4ac-4720-86a4-f9b7092b9f13 None None] MFA Rules not processed for user `f9a22f4b631f4fbf9b3d7795211eeccb`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:14.754910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7cb73bc7-b4ac-4720-86a4-f9b7092b9f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:14.826094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1043/2100] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:14.830531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110057f1-e853-48f4-a5e0-69daea6a9c3c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:14.843281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:14.894182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110057f1-e853-48f4-a5e0-69daea6a9c3c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:14.899390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623807f-0992-45e3-bed4-20f118e28eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:14.900075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623807f-0992-45e3-bed4-20f118e28eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:14.900360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623807f-0992-45e3-bed4-20f118e28eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:14.900733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623807f-0992-45e3-bed4-20f118e28eb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:14.901518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623807f-0992-45e3-bed4-20f118e28eb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:14.903432 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623807f-0992-45e3-bed4-20f118e28eb6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:14.928681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1058/2101] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:14 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:14.933942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb73bc7-b4ac-4720-86a4-f9b7092b9f13 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:14.964032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb73bc7-b4ac-4720-86a4-f9b7092b9f13 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:15.037020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91709e-21dc-4e96-885f-7e77e15ce663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:15.037799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91709e-21dc-4e96-885f-7e77e15ce663 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:15.038263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91709e-21dc-4e96-885f-7e77e15ce663 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:15.038537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91709e-21dc-4e96-885f-7e77e15ce663 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:15.039312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91709e-21dc-4e96-885f-7e77e15ce663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:15.042187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91709e-21dc-4e96-885f-7e77e15ce663 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:15.064327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1044/2102] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:14 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:15.068370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623807f-0992-45e3-bed4-20f118e28eb6 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:15.080383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623807f-0992-45e3-bed4-20f118e28eb6 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:15.085654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4e0da-7d81-49ef-b581-b4b93bdc5fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:15.086591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4e0da-7d81-49ef-b581-b4b93bdc5fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:15.087029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4e0da-7d81-49ef-b581-b4b93bdc5fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:15.087460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4e0da-7d81-49ef-b581-b4b93bdc5fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:15.088491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4e0da-7d81-49ef-b581-b4b93bdc5fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:15.090652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4e0da-7d81-49ef-b581-b4b93bdc5fb6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:15.103912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1059/2103] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:15.109001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91709e-21dc-4e96-885f-7e77e15ce663 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:15.122586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91709e-21dc-4e96-885f-7e77e15ce663 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:15.130696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b499082-2fb0-4c84-bd33-ee2574440520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:15.132128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b499082-2fb0-4c84-bd33-ee2574440520 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:15.132376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b499082-2fb0-4c84-bd33-ee2574440520 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:15.132688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b499082-2fb0-4c84-bd33-ee2574440520 admin admin] PATH_INFO: `/v3/projects/595afb337a2348518bfd838653e1aec4/users/d0a1e8990a6e48d08e452a4da8ef60f8/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:15.142618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:15.150902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b499082-2fb0-4c84-bd33-ee2574440520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=595afb337a2348518bfd838653e1aec4, user_id=d0a1e8990a6e48d08e452a4da8ef60f8, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:15.154336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b499082-2fb0-4c84-bd33-ee2574440520 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:15.169711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1045/2104] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:15 2022] PUT /identity/v3/projects/595afb337a2348518bfd838653e1aec4/users/d0a1e8990a6e48d08e452a4da8ef60f8/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:15.173640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4e0da-7d81-49ef-b581-b4b93bdc5fb6 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:15.183954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4e0da-7d81-49ef-b581-b4b93bdc5fb6 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:15.246234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a61c44-c19c-4736-a646-4e642a4a7506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:15.246911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a61c44-c19c-4736-a646-4e642a4a7506 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:15.247199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a61c44-c19c-4736-a646-4e642a4a7506 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:15.247507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a61c44-c19c-4736-a646-4e642a4a7506 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:15.248240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a61c44-c19c-4736-a646-4e642a4a7506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:15.249913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a61c44-c19c-4736-a646-4e642a4a7506 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:15.259919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1060/2105] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:50:15.264394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b499082-2fb0-4c84-bd33-ee2574440520 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:15.275068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b499082-2fb0-4c84-bd33-ee2574440520 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:15.280514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e600049-6d3b-44c1-8e74-181b9f1cea0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:15.281449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e600049-6d3b-44c1-8e74-181b9f1cea0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:15.281737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e600049-6d3b-44c1-8e74-181b9f1cea0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:15.282044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e600049-6d3b-44c1-8e74-181b9f1cea0f admin admin] PATH_INFO: `/v3/projects/595afb337a2348518bfd838653e1aec4/users/d0a1e8990a6e48d08e452a4da8ef60f8/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:15.284094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e600049-6d3b-44c1-8e74-181b9f1cea0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=595afb337a2348518bfd838653e1aec4, user_id=d0a1e8990a6e48d08e452a4da8ef60f8, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:15.286759 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e600049-6d3b-44c1-8e74-181b9f1cea0f admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:15.297293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1046/2106] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:15 2022] PUT /identity/v3/projects/595afb337a2348518bfd838653e1aec4/users/d0a1e8990a6e48d08e452a4da8ef60f8/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:15.307652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090279b-e003-4081-93bf-d5b04442d899 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:15.307943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090279b-e003-4081-93bf-d5b04442d899 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:15.308252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090279b-e003-4081-93bf-d5b04442d899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:15.334379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-7090279b-e003-4081-93bf-d5b04442d899 None None] MFA Rules not processed for user `d0a1e8990a6e48d08e452a4da8ef60f8`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:15.335570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-7090279b-e003-4081-93bf-d5b04442d899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:15.419574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1061/2107] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:15.430749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e600049-6d3b-44c1-8e74-181b9f1cea0f admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:15.432809 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7e600049-6d3b-44c1-8e74-181b9f1cea0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:15.491352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e600049-6d3b-44c1-8e74-181b9f1cea0f admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:15.495564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9c3d4-2cb5-4ab4-bf41-6b81833d6a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:15.496269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c3d4-2cb5-4ab4-bf41-6b81833d6a66 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:15.496495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c3d4-2cb5-4ab4-bf41-6b81833d6a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:15.496801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9c3d4-2cb5-4ab4-bf41-6b81833d6a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:15.506126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9c3d4-2cb5-4ab4-bf41-6b81833d6a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:15.507561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9c3d4-2cb5-4ab4-bf41-6b81833d6a66 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:15.517979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1047/2108] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:15.589689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090279b-e003-4081-93bf-d5b04442d899 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:15.599148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090279b-e003-4081-93bf-d5b04442d899 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:15.602756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311eb3d-a4f9-4c74-a265-fbc3718403c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:15.603390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311eb3d-a4f9-4c74-a265-fbc3718403c7 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:15.603673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311eb3d-a4f9-4c74-a265-fbc3718403c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:15.603950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311eb3d-a4f9-4c74-a265-fbc3718403c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:15.608592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:15.661474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311eb3d-a4f9-4c74-a265-fbc3718403c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:15.663138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311eb3d-a4f9-4c74-a265-fbc3718403c7 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:15.673138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1062/2109] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:17.316106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9c3d4-2cb5-4ab4-bf41-6b81833d6a66 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:17.325044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9c3d4-2cb5-4ab4-bf41-6b81833d6a66 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:17.328758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf110989-34c5-4848-a5c5-acebad8c1e44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:17.329331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf110989-34c5-4848-a5c5-acebad8c1e44 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:17.329609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf110989-34c5-4848-a5c5-acebad8c1e44 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:17.329883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf110989-34c5-4848-a5c5-acebad8c1e44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:17.331656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-cf110989-34c5-4848-a5c5-acebad8c1e44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:17.400196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf110989-34c5-4848-a5c5-acebad8c1e44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:17.402349 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf110989-34c5-4848-a5c5-acebad8c1e44 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:17.411389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1048/2110] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:50:19.584327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311eb3d-a4f9-4c74-a265-fbc3718403c7 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:19.585876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6311eb3d-a4f9-4c74-a265-fbc3718403c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:19.652522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311eb3d-a4f9-4c74-a265-fbc3718403c7 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:19.657177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:19.658056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:19.658344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:19.658623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] PATH_INFO: `/v3/users/326fa51732fd4105a49a7a24eb9fb1d1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:19.659820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=326fa51732fd4105a49a7a24eb9fb1d1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:19.661777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:19.721552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326fa51732fd4105a49a7a24eb9fb1d1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:19.734191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326fa51732fd4105a49a7a24eb9fb1d1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:19.734870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326fa51732fd4105a49a7a24eb9fb1d1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:19.735476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326fa51732fd4105a49a7a24eb9fb1d1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:19.742793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326fa51732fd4105a49a7a24eb9fb1d1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:19.755562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1063/2111] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:19 2022] DELETE /identity/v3/users/326fa51732fd4105a49a7a24eb9fb1d1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:19.760066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f034b1-24b8-4d85-af72-7a99df77eed3 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:19.760675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f034b1-24b8-4d85-af72-7a99df77eed3 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:19.760675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f034b1-24b8-4d85-af72-7a99df77eed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:19.778051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-12f034b1-24b8-4d85-af72-7a99df77eed3 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:19.779292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-12f034b1-24b8-4d85-af72-7a99df77eed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:19.837555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1049/2112] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:19.846980 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:19.848400 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:19.877139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:19.907216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095795a-c62d-45d1-b137-cdd2f166cdaf admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:19.910932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fa180-0d2e-4406-8511-8f08b312b5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:19.911579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fa180-0d2e-4406-8511-8f08b312b5cf service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:19.911865 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fa180-0d2e-4406-8511-8f08b312b5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:19.912151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fa180-0d2e-4406-8511-8f08b312b5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:19.922199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fa180-0d2e-4406-8511-8f08b312b5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:19.923626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fa180-0d2e-4406-8511-8f08b312b5cf service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:19.935197 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1064/2113] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:20.287351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f034b1-24b8-4d85-af72-7a99df77eed3 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:20.288915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-12f034b1-24b8-4d85-af72-7a99df77eed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:20.304492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f034b1-24b8-4d85-af72-7a99df77eed3 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:20.309811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:20.310685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:20.310969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:20.311247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] PATH_INFO: `/v3/projects/0d6eca66f9dd41128e5416bee1e9fcdc` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:20.312503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6eca66f9dd41128e5416bee1e9fcdc)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:20.314463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:20.326714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:20.363183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6eca66f9dd41128e5416bee1e9fcdc', 'request_id': 'req-0468e3dd-ea14-461f-9bbd-8a1003934249'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.369415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6eca66f9dd41128e5416bee1e9fcdc', 'request_id': 'req-0468e3dd-ea14-461f-9bbd-8a1003934249'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.376788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6eca66f9dd41128e5416bee1e9fcdc', 'request_id': 'req-0468e3dd-ea14-461f-9bbd-8a1003934249'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.387379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] The token cache is being invalidate because project 0d6eca66f9dd41128e5416bee1e9fcdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:20.387660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.388277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.389812 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1050/2114] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:20 2022] DELETE /identity/v3/projects/0d6eca66f9dd41128e5416bee1e9fcdc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:20.393866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fa180-0d2e-4406-8511-8f08b312b5cf service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:20.404543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fa180-0d2e-4406-8511-8f08b312b5cf service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:20.459250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:20.460056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:20.460356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:20.460635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] PATH_INFO: `/v3/users/64537005577044d788028d7dcc74e6c8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:20.461839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64537005577044d788028d7dcc74e6c8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:20.464025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:20.512280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64537005577044d788028d7dcc74e6c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.524395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64537005577044d788028d7dcc74e6c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.525114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64537005577044d788028d7dcc74e6c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.525726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64537005577044d788028d7dcc74e6c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.537092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64537005577044d788028d7dcc74e6c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.550323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1065/2115] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:20 2022] DELETE /identity/v3/users/64537005577044d788028d7dcc74e6c8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:20.828253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:20.829826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:20.843940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468e3dd-ea14-461f-9bbd-8a1003934249 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:20.918740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:20.919614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:20.919929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:20.920216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] PATH_INFO: `/v3/projects/a4c16ec3efe947d9b329abafaa030c08` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:20.921484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4c16ec3efe947d9b329abafaa030c08)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:20.923944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:20.987472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4c16ec3efe947d9b329abafaa030c08', 'request_id': 'req-7672d880-0371-4724-9c67-b5e8a8ad34d4'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:20.993980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4c16ec3efe947d9b329abafaa030c08', 'request_id': 'req-7672d880-0371-4724-9c67-b5e8a8ad34d4'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.002691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4c16ec3efe947d9b329abafaa030c08', 'request_id': 'req-7672d880-0371-4724-9c67-b5e8a8ad34d4'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.013686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] The token cache is being invalidate because project a4c16ec3efe947d9b329abafaa030c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:21.014050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.014729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.016265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1051/2116] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:20 2022] DELETE /identity/v3/projects/a4c16ec3efe947d9b329abafaa030c08 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:21.020222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:21.021773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:21.035811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99a590-cb3a-4104-9fe4-f5df92d8a396 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:21.084693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:21.101885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:21.102920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:21.103184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:21.103465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] PATH_INFO: `/v3/users/60c82864b86f4a8faa4d850dd57b9064` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:21.105543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=60c82864b86f4a8faa4d850dd57b9064)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:21.107481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:21.174473 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60c82864b86f4a8faa4d850dd57b9064'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.189856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60c82864b86f4a8faa4d850dd57b9064'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.190598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60c82864b86f4a8faa4d850dd57b9064'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.191772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60c82864b86f4a8faa4d850dd57b9064'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.205188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a1fb1f00-79c4-4477-abfb-585f99476bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60c82864b86f4a8faa4d850dd57b9064'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.231634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1066/2117] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:21 2022] DELETE /identity/v3/users/60c82864b86f4a8faa4d850dd57b9064 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:21.736520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:21.738188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:21.753597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672d880-0371-4724-9c67-b5e8a8ad34d4 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:21.789751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:21.842956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:21.843846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:21.844127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:21.844401 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] PATH_INFO: `/v3/projects/72957804b3464427bda227e4f2d98d9d` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:21.845653 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72957804b3464427bda227e4f2d98d9d)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:21.847588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:21.899973 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72957804b3464427bda227e4f2d98d9d', 'request_id': 'req-70361c99-147d-49fb-8554-415fcffd4b61'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.907926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72957804b3464427bda227e4f2d98d9d', 'request_id': 'req-70361c99-147d-49fb-8554-415fcffd4b61'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.915663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72957804b3464427bda227e4f2d98d9d', 'request_id': 'req-70361c99-147d-49fb-8554-415fcffd4b61'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.927094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] The token cache is being invalidate because project 72957804b3464427bda227e4f2d98d9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:21.927401 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.928030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-70361c99-147d-49fb-8554-415fcffd4b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:21.929600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1052/2118] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:21 2022] DELETE /identity/v3/projects/72957804b3464427bda227e4f2d98d9d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:21.936213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18190e8-df67-472a-90fe-99921d37377e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:21.936508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18190e8-df67-472a-90fe-99921d37377e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:21.936820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18190e8-df67-472a-90fe-99921d37377e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:21.954213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b18190e8-df67-472a-90fe-99921d37377e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:21.955402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b18190e8-df67-472a-90fe-99921d37377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:22.035900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1067/2119] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:22.044891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bdd42-ab08-49a6-ae21-1f77ac88d4f4 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.045315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bdd42-ab08-49a6-ae21-1f77ac88d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.045701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bdd42-ab08-49a6-ae21-1f77ac88d4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.064621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-200bdd42-ab08-49a6-ae21-1f77ac88d4f4 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:22.066006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-200bdd42-ab08-49a6-ae21-1f77ac88d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:22.078979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1053/2120] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:22.083056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18190e8-df67-472a-90fe-99921d37377e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.093986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18190e8-df67-472a-90fe-99921d37377e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.098740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038926b-08e4-4349-9e5c-f17da8947d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.099466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038926b-08e4-4349-9e5c-f17da8947d4e admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.099750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038926b-08e4-4349-9e5c-f17da8947d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.100026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038926b-08e4-4349-9e5c-f17da8947d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.100798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038926b-08e4-4349-9e5c-f17da8947d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:22.101318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038926b-08e4-4349-9e5c-f17da8947d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.102946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038926b-08e4-4349-9e5c-f17da8947d4e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.115618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1068/2121] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:50:22.119781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bdd42-ab08-49a6-ae21-1f77ac88d4f4 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.129718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bdd42-ab08-49a6-ae21-1f77ac88d4f4 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.134616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107cb12-da3c-4cc4-a83d-f74c911b0e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.136223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107cb12-da3c-4cc4-a83d-f74c911b0e90 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.136696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107cb12-da3c-4cc4-a83d-f74c911b0e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.137057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107cb12-da3c-4cc4-a83d-f74c911b0e90 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.137860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107cb12-da3c-4cc4-a83d-f74c911b0e90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.139915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107cb12-da3c-4cc4-a83d-f74c911b0e90 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.166720 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1054/2122] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:22 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:22.171156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038926b-08e4-4349-9e5c-f17da8947d4e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.181826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038926b-08e4-4349-9e5c-f17da8947d4e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.248732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:22.265317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d7f90-5f0e-415a-912c-2c258b768729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.266016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d7f90-5f0e-415a-912c-2c258b768729 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.267089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d7f90-5f0e-415a-912c-2c258b768729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.267089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d7f90-5f0e-415a-912c-2c258b768729 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.267793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d7f90-5f0e-415a-912c-2c258b768729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.271593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d7f90-5f0e-415a-912c-2c258b768729 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.304626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1069/2123] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:22 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:22.306251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107cb12-da3c-4cc4-a83d-f74c911b0e90 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.317405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107cb12-da3c-4cc4-a83d-f74c911b0e90 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.322175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e48d61-b013-4cb0-8865-e1919cc60ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.323708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e48d61-b013-4cb0-8865-e1919cc60ddb admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.324259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e48d61-b013-4cb0-8865-e1919cc60ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.324652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e48d61-b013-4cb0-8865-e1919cc60ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.325574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e48d61-b013-4cb0-8865-e1919cc60ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.327222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e48d61-b013-4cb0-8865-e1919cc60ddb admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.338582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1055/2124] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:22.343419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d7f90-5f0e-415a-912c-2c258b768729 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.353653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d7f90-5f0e-415a-912c-2c258b768729 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.358358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e97a20-6703-4386-b57b-d96ebbd64e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.359017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e97a20-6703-4386-b57b-d96ebbd64e2e admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.359300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e97a20-6703-4386-b57b-d96ebbd64e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.359581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e97a20-6703-4386-b57b-d96ebbd64e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.360340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e97a20-6703-4386-b57b-d96ebbd64e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.362070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e97a20-6703-4386-b57b-d96ebbd64e2e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.373411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1070/2125] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:22.380360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e48d61-b013-4cb0-8865-e1919cc60ddb admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.393196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e48d61-b013-4cb0-8865-e1919cc60ddb admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.398389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8d858-e266-4b2d-bbc1-94e72e5c85ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.399373 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8d858-e266-4b2d-bbc1-94e72e5c85ab admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.399716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8d858-e266-4b2d-bbc1-94e72e5c85ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.400057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8d858-e266-4b2d-bbc1-94e72e5c85ab admin admin] PATH_INFO: `/v3/projects/24c6167d55464b5c8ffd3a0a1f2069e6/users/8ac3441ed3904ee9a3ca77135ebfc45a/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.415471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8d858-e266-4b2d-bbc1-94e72e5c85ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c6167d55464b5c8ffd3a0a1f2069e6, user_id=8ac3441ed3904ee9a3ca77135ebfc45a, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.418194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8d858-e266-4b2d-bbc1-94e72e5c85ab admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.438302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1056/2126] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:22 2022] PUT /identity/v3/projects/24c6167d55464b5c8ffd3a0a1f2069e6/users/8ac3441ed3904ee9a3ca77135ebfc45a/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:22.450159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078cdf-eb41-43a5-8598-1a8ddaa8eee4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.450503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078cdf-eb41-43a5-8598-1a8ddaa8eee4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.450815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078cdf-eb41-43a5-8598-1a8ddaa8eee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.491304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-44078cdf-eb41-43a5-8598-1a8ddaa8eee4 None None] MFA Rules not processed for user `8ac3441ed3904ee9a3ca77135ebfc45a`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:22.492592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-44078cdf-eb41-43a5-8598-1a8ddaa8eee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:22.583114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1071/2127] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:22.587448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8d858-e266-4b2d-bbc1-94e72e5c85ab admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.665938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:22.672329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8d858-e266-4b2d-bbc1-94e72e5c85ab admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.678377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5504631-0ac2-419f-9a70-67eddfaa94bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.679049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5504631-0ac2-419f-9a70-67eddfaa94bd admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.679515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5504631-0ac2-419f-9a70-67eddfaa94bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.679980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5504631-0ac2-419f-9a70-67eddfaa94bd admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.680823 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5504631-0ac2-419f-9a70-67eddfaa94bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.683234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5504631-0ac2-419f-9a70-67eddfaa94bd admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.711661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1057/2128] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:22 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:22.715967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44078cdf-eb41-43a5-8598-1a8ddaa8eee4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.726873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44078cdf-eb41-43a5-8598-1a8ddaa8eee4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.785681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd455304-0332-4a5d-9a31-a9784f53ff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.786353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd455304-0332-4a5d-9a31-a9784f53ff64 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.786635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd455304-0332-4a5d-9a31-a9784f53ff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.786913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd455304-0332-4a5d-9a31-a9784f53ff64 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.787624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd455304-0332-4a5d-9a31-a9784f53ff64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.789575 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd455304-0332-4a5d-9a31-a9784f53ff64 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.813333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1072/2129] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:22 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:22.814565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5504631-0ac2-419f-9a70-67eddfaa94bd admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.825786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5504631-0ac2-419f-9a70-67eddfaa94bd admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.830724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b4bea-0fed-40e0-82d2-e0bd4b3b36d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.833513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b4bea-0fed-40e0-82d2-e0bd4b3b36d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.833807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b4bea-0fed-40e0-82d2-e0bd4b3b36d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.834093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b4bea-0fed-40e0-82d2-e0bd4b3b36d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.834842 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b4bea-0fed-40e0-82d2-e0bd4b3b36d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.836474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b4bea-0fed-40e0-82d2-e0bd4b3b36d8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.848792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1058/2130] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:50:22.853341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd455304-0332-4a5d-9a31-a9784f53ff64 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.867166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd455304-0332-4a5d-9a31-a9784f53ff64 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:22.875563 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06c104-c17c-46c0-8651-979443d66807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:22.877027 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06c104-c17c-46c0-8651-979443d66807 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:22.877388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06c104-c17c-46c0-8651-979443d66807 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:22.877741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06c104-c17c-46c0-8651-979443d66807 admin admin] PATH_INFO: `/v3/projects/f5765e08d61545dcbb46c97c760e0010/users/b365497a0dd74bb5b9cadcdef97b7839/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:22.897310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06c104-c17c-46c0-8651-979443d66807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5765e08d61545dcbb46c97c760e0010, user_id=b365497a0dd74bb5b9cadcdef97b7839, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:22.900533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06c104-c17c-46c0-8651-979443d66807 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:22.918629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1073/2131] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:22 2022] PUT /identity/v3/projects/f5765e08d61545dcbb46c97c760e0010/users/b365497a0dd74bb5b9cadcdef97b7839/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:22.922582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b4bea-0fed-40e0-82d2-e0bd4b3b36d8 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:22.939896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b4bea-0fed-40e0-82d2-e0bd4b3b36d8 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:23.027090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915724fa-c491-43a9-a2bd-89915a5aa0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:23.027956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915724fa-c491-43a9-a2bd-89915a5aa0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:23.028299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915724fa-c491-43a9-a2bd-89915a5aa0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:23.028604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915724fa-c491-43a9-a2bd-89915a5aa0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:23.029572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915724fa-c491-43a9-a2bd-89915a5aa0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:23.031185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915724fa-c491-43a9-a2bd-89915a5aa0f8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:23.044617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1059/2132] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:23.047853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06c104-c17c-46c0-8651-979443d66807 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:23.058676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06c104-c17c-46c0-8651-979443d66807 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:23.065193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c268770-746c-4662-8f50-1256acf6ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:23.066175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c268770-746c-4662-8f50-1256acf6ecf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:23.066539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c268770-746c-4662-8f50-1256acf6ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:23.066893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c268770-746c-4662-8f50-1256acf6ecf2 admin admin] PATH_INFO: `/v3/projects/f5765e08d61545dcbb46c97c760e0010/users/b365497a0dd74bb5b9cadcdef97b7839/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:23.069196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c268770-746c-4662-8f50-1256acf6ecf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5765e08d61545dcbb46c97c760e0010, user_id=b365497a0dd74bb5b9cadcdef97b7839, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:23.072346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c268770-746c-4662-8f50-1256acf6ecf2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:23.084935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1074/2133] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:23 2022] PUT /identity/v3/projects/f5765e08d61545dcbb46c97c760e0010/users/b365497a0dd74bb5b9cadcdef97b7839/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:23.097023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c90e05-4a71-4298-b13d-b6b4ae17ad0a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:23.097311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c90e05-4a71-4298-b13d-b6b4ae17ad0a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:23.097624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c90e05-4a71-4298-b13d-b6b4ae17ad0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:23.127341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-75c90e05-4a71-4298-b13d-b6b4ae17ad0a None None] MFA Rules not processed for user `b365497a0dd74bb5b9cadcdef97b7839`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:23.128646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-75c90e05-4a71-4298-b13d-b6b4ae17ad0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:23.232791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1060/2134] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:23.244009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c268770-746c-4662-8f50-1256acf6ecf2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:23.246228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4c268770-746c-4662-8f50-1256acf6ecf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:23.263325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:23.311590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c268770-746c-4662-8f50-1256acf6ecf2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:23.315215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2f127-66b6-482e-8987-888ab996571a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:23.315905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2f127-66b6-482e-8987-888ab996571a service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:23.316193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2f127-66b6-482e-8987-888ab996571a service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:23.316469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2f127-66b6-482e-8987-888ab996571a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:23.327920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2f127-66b6-482e-8987-888ab996571a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:23.330053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2f127-66b6-482e-8987-888ab996571a service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:23.343909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1075/2135] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:50:23.475171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c90e05-4a71-4298-b13d-b6b4ae17ad0a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:23.528797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:23.544252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c90e05-4a71-4298-b13d-b6b4ae17ad0a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:23.549430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:23.550311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:23.550589 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:23.550865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] PATH_INFO: `/v3/users/8ac3441ed3904ee9a3ca77135ebfc45a` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:23.552330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac3441ed3904ee9a3ca77135ebfc45a)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:23.554385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:23.605056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac3441ed3904ee9a3ca77135ebfc45a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:23.616877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac3441ed3904ee9a3ca77135ebfc45a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:23.626348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac3441ed3904ee9a3ca77135ebfc45a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:23.626992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac3441ed3904ee9a3ca77135ebfc45a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:23.634698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac3441ed3904ee9a3ca77135ebfc45a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:23.640742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1061/2136] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:23 2022] DELETE /identity/v3/users/8ac3441ed3904ee9a3ca77135ebfc45a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:23.645463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487736b-3b60-4bbd-b3b0-c97dcc08d5a8 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:23.645761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487736b-3b60-4bbd-b3b0-c97dcc08d5a8 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:23.646044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487736b-3b60-4bbd-b3b0-c97dcc08d5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:23.669382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b487736b-3b60-4bbd-b3b0-c97dcc08d5a8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:23.671202 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b487736b-3b60-4bbd-b3b0-c97dcc08d5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:23.735566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1076/2137] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:23.745515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:23.746924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:23.803907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d57de-7321-452f-9998-68f48d2b9bf8 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:23.808209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640f2cc-5926-4c44-8669-5bf8d8e67b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:23.808910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640f2cc-5926-4c44-8669-5bf8d8e67b4e service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:23.809193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640f2cc-5926-4c44-8669-5bf8d8e67b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:23.809469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640f2cc-5926-4c44-8669-5bf8d8e67b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:23.818385 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640f2cc-5926-4c44-8669-5bf8d8e67b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:23.819872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640f2cc-5926-4c44-8669-5bf8d8e67b4e service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:23.830111 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1062/2138] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:24.114190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487736b-3b60-4bbd-b3b0-c97dcc08d5a8 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:24.115777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b487736b-3b60-4bbd-b3b0-c97dcc08d5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:24.151818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487736b-3b60-4bbd-b3b0-c97dcc08d5a8 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:24.170773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:24.171660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:24.171940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:24.172397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] PATH_INFO: `/v3/projects/24c6167d55464b5c8ffd3a0a1f2069e6` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:24.173642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c6167d55464b5c8ffd3a0a1f2069e6)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:24.175553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:24.226516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c6167d55464b5c8ffd3a0a1f2069e6', 'request_id': 'req-843323a9-9d39-4dc0-bf1a-75254c91b5ec'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.238391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c6167d55464b5c8ffd3a0a1f2069e6', 'request_id': 'req-843323a9-9d39-4dc0-bf1a-75254c91b5ec'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.249062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c6167d55464b5c8ffd3a0a1f2069e6', 'request_id': 'req-843323a9-9d39-4dc0-bf1a-75254c91b5ec'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.262358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] The token cache is being invalidate because project 24c6167d55464b5c8ffd3a0a1f2069e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:24.262902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.263647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.265878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1077/2139] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:24 2022] DELETE /identity/v3/projects/24c6167d55464b5c8ffd3a0a1f2069e6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:24.269778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640f2cc-5926-4c44-8669-5bf8d8e67b4e service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:24.281764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640f2cc-5926-4c44-8669-5bf8d8e67b4e service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:24.361471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:24.362828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:24.364137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:24.364965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] PATH_INFO: `/v3/users/b365497a0dd74bb5b9cadcdef97b7839` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:24.366595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b365497a0dd74bb5b9cadcdef97b7839)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:24.368891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:24.388337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:24.448235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b365497a0dd74bb5b9cadcdef97b7839'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.464859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b365497a0dd74bb5b9cadcdef97b7839'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.474896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b365497a0dd74bb5b9cadcdef97b7839'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.475737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b365497a0dd74bb5b9cadcdef97b7839'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.484989 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5bc38398-f130-4a36-9d70-32b0e12f11c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b365497a0dd74bb5b9cadcdef97b7839'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:24.492853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1063/2140] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:24 2022] DELETE /identity/v3/users/b365497a0dd74bb5b9cadcdef97b7839 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:24.849463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:24.851146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:24.865370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843323a9-9d39-4dc0-bf1a-75254c91b5ec admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:24.921457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:24.944078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:24.945115 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:24.945386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:24.945667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] PATH_INFO: `/v3/projects/f5765e08d61545dcbb46c97c760e0010` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:24.946856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5765e08d61545dcbb46c97c760e0010)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:24.949552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:24.999740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5765e08d61545dcbb46c97c760e0010', 'request_id': 'req-5fd9e76c-a811-450d-b312-ae3cb838241d'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:25.012756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5765e08d61545dcbb46c97c760e0010', 'request_id': 'req-5fd9e76c-a811-450d-b312-ae3cb838241d'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:25.021539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5765e08d61545dcbb46c97c760e0010', 'request_id': 'req-5fd9e76c-a811-450d-b312-ae3cb838241d'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:25.035185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] The token cache is being invalidate because project f5765e08d61545dcbb46c97c760e0010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:25.035480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:25.036145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-5fd9e76c-a811-450d-b312-ae3cb838241d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:25.037829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1078/2141] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:24 2022] DELETE /identity/v3/projects/f5765e08d61545dcbb46c97c760e0010 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:25.045055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671d121-ef41-4519-90f8-9be76f62a7a7 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.045340 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671d121-ef41-4519-90f8-9be76f62a7a7 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.045621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671d121-ef41-4519-90f8-9be76f62a7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.063480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-b671d121-ef41-4519-90f8-9be76f62a7a7 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:25.064719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b671d121-ef41-4519-90f8-9be76f62a7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:25.125646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1064/2142] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:25.133596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23289a1f-7c1d-4f85-bd46-8b7ef74aeed9 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.134104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23289a1f-7c1d-4f85-bd46-8b7ef74aeed9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.134217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23289a1f-7c1d-4f85-bd46-8b7ef74aeed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.155031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-23289a1f-7c1d-4f85-bd46-8b7ef74aeed9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:25.156518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-23289a1f-7c1d-4f85-bd46-8b7ef74aeed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:25.165526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1079/2143] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:25.169926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671d121-ef41-4519-90f8-9be76f62a7a7 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:25.180816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671d121-ef41-4519-90f8-9be76f62a7a7 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:25.185907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc0ee-bb8a-4594-ab54-e64a79bd5a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:25.186573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc0ee-bb8a-4594-ab54-e64a79bd5a5a admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.186887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc0ee-bb8a-4594-ab54-e64a79bd5a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.187169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc0ee-bb8a-4594-ab54-e64a79bd5a5a admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.187910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dc0ee-bb8a-4594-ab54-e64a79bd5a5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:25.188225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dc0ee-bb8a-4594-ab54-e64a79bd5a5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:25.189883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dc0ee-bb8a-4594-ab54-e64a79bd5a5a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:25.202577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1065/2144] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:50:25.206871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23289a1f-7c1d-4f85-bd46-8b7ef74aeed9 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:25.216784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23289a1f-7c1d-4f85-bd46-8b7ef74aeed9 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:25.221489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d80ad8-08da-42b1-ae2e-e258e6a86702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:25.222131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d80ad8-08da-42b1-ae2e-e258e6a86702 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.222417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d80ad8-08da-42b1-ae2e-e258e6a86702 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.222690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d80ad8-08da-42b1-ae2e-e258e6a86702 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.223379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d80ad8-08da-42b1-ae2e-e258e6a86702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:25.225336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d80ad8-08da-42b1-ae2e-e258e6a86702 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:25.256158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1080/2145] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:25 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:25.257248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc0ee-bb8a-4594-ab54-e64a79bd5a5a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:25.267780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc0ee-bb8a-4594-ab54-e64a79bd5a5a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:25.334191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59001759-1045-4d6d-8d62-a9b63c7584fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:25.334881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59001759-1045-4d6d-8d62-a9b63c7584fa admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.335166 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59001759-1045-4d6d-8d62-a9b63c7584fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.335445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59001759-1045-4d6d-8d62-a9b63c7584fa admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.336149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59001759-1045-4d6d-8d62-a9b63c7584fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:25.338602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59001759-1045-4d6d-8d62-a9b63c7584fa admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:25.362441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1066/2146] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:25 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:25.367113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d80ad8-08da-42b1-ae2e-e258e6a86702 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:25.381402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d80ad8-08da-42b1-ae2e-e258e6a86702 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:25.387736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a0f5b-54eb-46fa-9458-eb8a505242d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:25.392749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a0f5b-54eb-46fa-9458-eb8a505242d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.393265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a0f5b-54eb-46fa-9458-eb8a505242d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.393738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a0f5b-54eb-46fa-9458-eb8a505242d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.394674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a0f5b-54eb-46fa-9458-eb8a505242d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:25.400248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a0f5b-54eb-46fa-9458-eb8a505242d4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:25.420421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1081/2147] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:25.421392 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59001759-1045-4d6d-8d62-a9b63c7584fa admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:25.433359 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59001759-1045-4d6d-8d62-a9b63c7584fa admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:25.439101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dc652-24dc-45d2-8373-d400eb45e1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:25.439355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dc652-24dc-45d2-8373-d400eb45e1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.439641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dc652-24dc-45d2-8373-d400eb45e1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.439934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dc652-24dc-45d2-8373-d400eb45e1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.440810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dc652-24dc-45d2-8373-d400eb45e1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:25.442416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dc652-24dc-45d2-8373-d400eb45e1c1 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:25.445607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:25.457059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1067/2148] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:25.461150 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a0f5b-54eb-46fa-9458-eb8a505242d4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:25.474472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a0f5b-54eb-46fa-9458-eb8a505242d4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:25.479724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120c34d-1598-4ecb-98bd-b9be798e0e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:25.480292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120c34d-1598-4ecb-98bd-b9be798e0e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.480599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120c34d-1598-4ecb-98bd-b9be798e0e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.480883 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120c34d-1598-4ecb-98bd-b9be798e0e2d admin admin] PATH_INFO: `/v3/projects/e318cfdc9de345148828c09515cecdc1/users/7faba3540b1b4ef5a533ab0ff8e1cca3/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.496845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120c34d-1598-4ecb-98bd-b9be798e0e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e318cfdc9de345148828c09515cecdc1, user_id=7faba3540b1b4ef5a533ab0ff8e1cca3, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:25.499669 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120c34d-1598-4ecb-98bd-b9be798e0e2d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:25.518053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1082/2149] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:25 2022] PUT /identity/v3/projects/e318cfdc9de345148828c09515cecdc1/users/7faba3540b1b4ef5a533ab0ff8e1cca3/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:25.523336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c473f38-0b8e-40d4-a346-54d19041d993 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.524200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c473f38-0b8e-40d4-a346-54d19041d993 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.524200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c473f38-0b8e-40d4-a346-54d19041d993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.540741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8c473f38-0b8e-40d4-a346-54d19041d993 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:25.542318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8c473f38-0b8e-40d4-a346-54d19041d993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:25.611382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1068/2150] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:25.621929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120c34d-1598-4ecb-98bd-b9be798e0e2d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:25.623692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3120c34d-1598-4ecb-98bd-b9be798e0e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:25.684819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120c34d-1598-4ecb-98bd-b9be798e0e2d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:25.688352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f762873-12f6-479f-b72e-4b7f2500fe82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:25.689029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f762873-12f6-479f-b72e-4b7f2500fe82 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:25.689318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f762873-12f6-479f-b72e-4b7f2500fe82 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:25.689592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f762873-12f6-479f-b72e-4b7f2500fe82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:25.699853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f762873-12f6-479f-b72e-4b7f2500fe82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:25.701364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f762873-12f6-479f-b72e-4b7f2500fe82 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:25.711654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1083/2151] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:26.251017 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c473f38-0b8e-40d4-a346-54d19041d993 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:26.252704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8c473f38-0b8e-40d4-a346-54d19041d993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:26.266153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c473f38-0b8e-40d4-a346-54d19041d993 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:26.271161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:26.272384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:26.272690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:26.273143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] PATH_INFO: `/v3/users/f9a22f4b631f4fbf9b3d7795211eeccb` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:26.274411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9a22f4b631f4fbf9b3d7795211eeccb)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:26.277502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:26.325926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9a22f4b631f4fbf9b3d7795211eeccb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:26.337849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9a22f4b631f4fbf9b3d7795211eeccb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:26.347565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9a22f4b631f4fbf9b3d7795211eeccb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:26.348273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9a22f4b631f4fbf9b3d7795211eeccb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:26.358778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9a22f4b631f4fbf9b3d7795211eeccb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:26.365885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1069/2152] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:26 2022] DELETE /identity/v3/users/f9a22f4b631f4fbf9b3d7795211eeccb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:26.370773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d80d6-b173-47a2-902d-c4bac87e60be None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:26.371238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d80d6-b173-47a2-902d-c4bac87e60be None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:26.371365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d80d6-b173-47a2-902d-c4bac87e60be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:26.390694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ff3d80d6-b173-47a2-902d-c4bac87e60be None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:26.391903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ff3d80d6-b173-47a2-902d-c4bac87e60be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:26.396130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:26.508395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1084/2153] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:26.524838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:26.528048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:26.619014 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f70e04-fa9e-4360-837c-6553c0c22a74 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:26.623940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b51e21-0c61-471b-9a1c-2767d92039da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:26.624705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b51e21-0c61-471b-9a1c-2767d92039da service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:26.624994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b51e21-0c61-471b-9a1c-2767d92039da service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:26.625280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b51e21-0c61-471b-9a1c-2767d92039da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:26.637091 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b51e21-0c61-471b-9a1c-2767d92039da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:26.638840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b51e21-0c61-471b-9a1c-2767d92039da service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:26.655080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1070/2154] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:27.049982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d80d6-b173-47a2-902d-c4bac87e60be None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:27.051501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ff3d80d6-b173-47a2-902d-c4bac87e60be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:27.080157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d80d6-b173-47a2-902d-c4bac87e60be None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:27.089673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:27.097692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:27.100740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:27.101303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] PATH_INFO: `/v3/projects/3bbadf83a57c496495289ad4cc93f135` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:27.110297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bbadf83a57c496495289ad4cc93f135)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:27.112527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:27.227690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bbadf83a57c496495289ad4cc93f135', 'request_id': 'req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.240885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bbadf83a57c496495289ad4cc93f135', 'request_id': 'req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.250539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bbadf83a57c496495289ad4cc93f135', 'request_id': 'req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.265042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] The token cache is being invalidate because project 3bbadf83a57c496495289ad4cc93f135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:27.265564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.266410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.271675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1085/2155] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:27 2022] DELETE /identity/v3/projects/3bbadf83a57c496495289ad4cc93f135 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:27.273110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b51e21-0c61-471b-9a1c-2767d92039da service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:27.286850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b51e21-0c61-471b-9a1c-2767d92039da service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:27.304077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:27.409426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:27.410323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:27.410642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:27.410875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] PATH_INFO: `/v3/users/d0a1e8990a6e48d08e452a4da8ef60f8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:27.412967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a1e8990a6e48d08e452a4da8ef60f8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:27.415732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:27.517606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a1e8990a6e48d08e452a4da8ef60f8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.541861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a1e8990a6e48d08e452a4da8ef60f8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.552230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a1e8990a6e48d08e452a4da8ef60f8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.552980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a1e8990a6e48d08e452a4da8ef60f8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.563094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-45f0fc4b-9864-4ad2-8cb7-667181cdaead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a1e8990a6e48d08e452a4da8ef60f8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:27.571487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1071/2156] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:27 2022] DELETE /identity/v3/users/d0a1e8990a6e48d08e452a4da8ef60f8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:27.989240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:27.990672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:28.014439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79912ba8-1e9c-4fa3-8eba-cf4afa433ca0 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:28.104156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:28.105180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.105466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.105743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] PATH_INFO: `/v3/projects/595afb337a2348518bfd838653e1aec4` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.106969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=595afb337a2348518bfd838653e1aec4)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:28.109767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:28.159544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:28.174799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '595afb337a2348518bfd838653e1aec4', 'request_id': 'req-3b0e6351-76a5-4aca-b901-8429c2306c3a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:28.185546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '595afb337a2348518bfd838653e1aec4', 'request_id': 'req-3b0e6351-76a5-4aca-b901-8429c2306c3a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:28.193877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '595afb337a2348518bfd838653e1aec4', 'request_id': 'req-3b0e6351-76a5-4aca-b901-8429c2306c3a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:28.217048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] The token cache is being invalidate because project 595afb337a2348518bfd838653e1aec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:28.217304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:28.217936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3b0e6351-76a5-4aca-b901-8429c2306c3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:28.222842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1086/2157] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:27 2022] DELETE /identity/v3/projects/595afb337a2348518bfd838653e1aec4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:28.224851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ca428-b848-4527-b484-0f746d364af8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.225355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ca428-b848-4527-b484-0f746d364af8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.225597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ca428-b848-4527-b484-0f746d364af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.250245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-015ca428-b848-4527-b484-0f746d364af8 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:28.251563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-015ca428-b848-4527-b484-0f746d364af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:28.325774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1072/2158] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:28.333729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d0f7e-83fa-4170-9df8-323b596a3f0d None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.334024 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d0f7e-83fa-4170-9df8-323b596a3f0d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.334315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d0f7e-83fa-4170-9df8-323b596a3f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.356230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-548d0f7e-83fa-4170-9df8-323b596a3f0d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:28.357573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-548d0f7e-83fa-4170-9df8-323b596a3f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:28.366215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1087/2159] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:28.370173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ca428-b848-4527-b484-0f746d364af8 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:28.380008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ca428-b848-4527-b484-0f746d364af8 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:28.384731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f749e-78d8-40de-aa71-9996f4b62fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:28.385571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f749e-78d8-40de-aa71-9996f4b62fce admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.385866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f749e-78d8-40de-aa71-9996f4b62fce admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.386121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f749e-78d8-40de-aa71-9996f4b62fce admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.386882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f749e-78d8-40de-aa71-9996f4b62fce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:28.387176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f749e-78d8-40de-aa71-9996f4b62fce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:28.388905 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f749e-78d8-40de-aa71-9996f4b62fce admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:28.405799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1073/2160] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:50:28.407015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d0f7e-83fa-4170-9df8-323b596a3f0d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:28.418263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d0f7e-83fa-4170-9df8-323b596a3f0d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:28.424261 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02259f61-3730-46ad-98d7-ebb8daf833e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:28.428636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02259f61-3730-46ad-98d7-ebb8daf833e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.428636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02259f61-3730-46ad-98d7-ebb8daf833e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.428636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02259f61-3730-46ad-98d7-ebb8daf833e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.428636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02259f61-3730-46ad-98d7-ebb8daf833e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:28.429949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02259f61-3730-46ad-98d7-ebb8daf833e2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:28.469938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f749e-78d8-40de-aa71-9996f4b62fce admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:28.475395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1088/2161] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:28 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:28.488952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f749e-78d8-40de-aa71-9996f4b62fce admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:28.531997 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:28.651029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7064ee-d56d-4dc7-9374-7056144f266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:28.651730 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7064ee-d56d-4dc7-9374-7056144f266b admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.652017 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7064ee-d56d-4dc7-9374-7056144f266b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.652302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7064ee-d56d-4dc7-9374-7056144f266b admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.653411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7064ee-d56d-4dc7-9374-7056144f266b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:28.655675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7064ee-d56d-4dc7-9374-7056144f266b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:28.693446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1074/2162] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:28 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:28.694694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02259f61-3730-46ad-98d7-ebb8daf833e2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:28.707177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02259f61-3730-46ad-98d7-ebb8daf833e2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:28.715469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c778ae-9c54-4330-9d00-a7dbfd7f876e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:28.716378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c778ae-9c54-4330-9d00-a7dbfd7f876e admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.716771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c778ae-9c54-4330-9d00-a7dbfd7f876e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.717055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c778ae-9c54-4330-9d00-a7dbfd7f876e admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.717805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c778ae-9c54-4330-9d00-a7dbfd7f876e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:28.719437 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c778ae-9c54-4330-9d00-a7dbfd7f876e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:28.735161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1089/2163] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:28.735679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7064ee-d56d-4dc7-9374-7056144f266b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:28.751000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7064ee-d56d-4dc7-9374-7056144f266b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:28.756066 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b1b40-5555-4363-934f-f123dfcd6067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:28.757163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b1b40-5555-4363-934f-f123dfcd6067 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.757464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b1b40-5555-4363-934f-f123dfcd6067 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.757741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b1b40-5555-4363-934f-f123dfcd6067 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.758477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b1b40-5555-4363-934f-f123dfcd6067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:28.760710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b1b40-5555-4363-934f-f123dfcd6067 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:28.776574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c778ae-9c54-4330-9d00-a7dbfd7f876e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:28.777657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1075/2164] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:28.796729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c778ae-9c54-4330-9d00-a7dbfd7f876e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:28.796729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc89c7-7457-40e4-85b6-78197c4b7212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:28.796729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc89c7-7457-40e4-85b6-78197c4b7212 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.797613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc89c7-7457-40e4-85b6-78197c4b7212 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.797613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc89c7-7457-40e4-85b6-78197c4b7212 admin admin] PATH_INFO: `/v3/projects/b3679689d28449bb9c8a5df37de4fb92/users/2bc0aa5115cb455c9576eceaa655fb93/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.817503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc89c7-7457-40e4-85b6-78197c4b7212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3679689d28449bb9c8a5df37de4fb92, user_id=2bc0aa5115cb455c9576eceaa655fb93, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:28.821086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc89c7-7457-40e4-85b6-78197c4b7212 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:28.842992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1090/2165] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:28 2022] PUT /identity/v3/projects/b3679689d28449bb9c8a5df37de4fb92/users/2bc0aa5115cb455c9576eceaa655fb93/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:28.854803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924afd4b-e06e-47fe-861d-2c4ad535bf34 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:28.855612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924afd4b-e06e-47fe-861d-2c4ad535bf34 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:28.855898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924afd4b-e06e-47fe-861d-2c4ad535bf34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:28.888764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-924afd4b-e06e-47fe-861d-2c4ad535bf34 None None] MFA Rules not processed for user `2bc0aa5115cb455c9576eceaa655fb93`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:28.889996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-924afd4b-e06e-47fe-861d-2c4ad535bf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:28.981446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1076/2166] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:28.986703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc89c7-7457-40e4-85b6-78197c4b7212 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:29.063912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc89c7-7457-40e4-85b6-78197c4b7212 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:29.069084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23e99-f9a1-4f47-9643-027f8a3fcfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:29.069884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23e99-f9a1-4f47-9643-027f8a3fcfc2 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:29.070168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23e99-f9a1-4f47-9643-027f8a3fcfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:29.070446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df23e99-f9a1-4f47-9643-027f8a3fcfc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:29.071147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df23e99-f9a1-4f47-9643-027f8a3fcfc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:29.073488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df23e99-f9a1-4f47-9643-027f8a3fcfc2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:29.109830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1091/2167] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:28 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:29.113919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924afd4b-e06e-47fe-861d-2c4ad535bf34 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:29.134245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924afd4b-e06e-47fe-861d-2c4ad535bf34 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:29.233179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30d5a0-63f0-4aaa-ab8d-233c41f306a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:29.233881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30d5a0-63f0-4aaa-ab8d-233c41f306a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:29.234166 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30d5a0-63f0-4aaa-ab8d-233c41f306a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:29.234446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30d5a0-63f0-4aaa-ab8d-233c41f306a4 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:29.235162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30d5a0-63f0-4aaa-ab8d-233c41f306a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:29.237282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30d5a0-63f0-4aaa-ab8d-233c41f306a4 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:29.271379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1077/2168] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:29 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:29.276935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23e99-f9a1-4f47-9643-027f8a3fcfc2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:29.312700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df23e99-f9a1-4f47-9643-027f8a3fcfc2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:29.318778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46ef8d-1390-4a80-b26d-6816fa7186ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:29.319615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46ef8d-1390-4a80-b26d-6816fa7186ad admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:29.319901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46ef8d-1390-4a80-b26d-6816fa7186ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:29.320171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46ef8d-1390-4a80-b26d-6816fa7186ad admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:29.321241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46ef8d-1390-4a80-b26d-6816fa7186ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:29.324181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f46ef8d-1390-4a80-b26d-6816fa7186ad admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:29.337643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1092/2169] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:29.342462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30d5a0-63f0-4aaa-ab8d-233c41f306a4 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:29.356976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30d5a0-63f0-4aaa-ab8d-233c41f306a4 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:29.361935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414806da-ff91-495b-b4e2-8f1231238e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:29.362845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414806da-ff91-495b-b4e2-8f1231238e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:29.363124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414806da-ff91-495b-b4e2-8f1231238e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:29.363399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414806da-ff91-495b-b4e2-8f1231238e97 admin admin] PATH_INFO: `/v3/projects/3f0fd62432f04569b068e432ac2d250e/users/3010644ebaac4fbab4996db86cb31a95/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:29.381690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414806da-ff91-495b-b4e2-8f1231238e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f0fd62432f04569b068e432ac2d250e, user_id=3010644ebaac4fbab4996db86cb31a95, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:29.385964 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414806da-ff91-495b-b4e2-8f1231238e97 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:29.389082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:29.413272 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1078/2170] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:29 2022] PUT /identity/v3/projects/3f0fd62432f04569b068e432ac2d250e/users/3010644ebaac4fbab4996db86cb31a95/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:29.421957 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46ef8d-1390-4a80-b26d-6816fa7186ad admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:29.434273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46ef8d-1390-4a80-b26d-6816fa7186ad admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:29.472903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:29.506635 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a3b5c-4b85-4f37-b5b3-c3f271b73ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:29.507345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3b5c-4b85-4f37-b5b3-c3f271b73ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:29.507625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3b5c-4b85-4f37-b5b3-c3f271b73ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:29.507896 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a3b5c-4b85-4f37-b5b3-c3f271b73ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:29.511874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a3b5c-4b85-4f37-b5b3-c3f271b73ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:29.512846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a3b5c-4b85-4f37-b5b3-c3f271b73ad0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:29.533808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1093/2171] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:29.535180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414806da-ff91-495b-b4e2-8f1231238e97 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:29.546423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414806da-ff91-495b-b4e2-8f1231238e97 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:29.551324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af444ad-1eb7-4c8c-8298-5b9a9b431206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:29.552289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af444ad-1eb7-4c8c-8298-5b9a9b431206 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:29.552880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af444ad-1eb7-4c8c-8298-5b9a9b431206 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:29.552989 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af444ad-1eb7-4c8c-8298-5b9a9b431206 admin admin] PATH_INFO: `/v3/projects/3f0fd62432f04569b068e432ac2d250e/users/3010644ebaac4fbab4996db86cb31a95/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:29.555142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af444ad-1eb7-4c8c-8298-5b9a9b431206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f0fd62432f04569b068e432ac2d250e, user_id=3010644ebaac4fbab4996db86cb31a95, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:29.558055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af444ad-1eb7-4c8c-8298-5b9a9b431206 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:29.571986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1079/2172] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:29 2022] PUT /identity/v3/projects/3f0fd62432f04569b068e432ac2d250e/users/3010644ebaac4fbab4996db86cb31a95/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:29.587264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e518aa5-02cf-4c3b-b132-aa0cc0f60993 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:29.589030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e518aa5-02cf-4c3b-b132-aa0cc0f60993 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:29.590051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e518aa5-02cf-4c3b-b132-aa0cc0f60993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:29.628612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-6e518aa5-02cf-4c3b-b132-aa0cc0f60993 None None] MFA Rules not processed for user `3010644ebaac4fbab4996db86cb31a95`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:29.629844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6e518aa5-02cf-4c3b-b132-aa0cc0f60993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:29.748155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1094/2173] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:29.762084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af444ad-1eb7-4c8c-8298-5b9a9b431206 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:29.763826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8af444ad-1eb7-4c8c-8298-5b9a9b431206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:29.903103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af444ad-1eb7-4c8c-8298-5b9a9b431206 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:29.912685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17147391-c91e-41be-aedc-675cf3dc11e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:29.913387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17147391-c91e-41be-aedc-675cf3dc11e1 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:29.913676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17147391-c91e-41be-aedc-675cf3dc11e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:29.913960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17147391-c91e-41be-aedc-675cf3dc11e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:29.929390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17147391-c91e-41be-aedc-675cf3dc11e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:29.931039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17147391-c91e-41be-aedc-675cf3dc11e1 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:29.951348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1080/2174] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:30.457426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e518aa5-02cf-4c3b-b132-aa0cc0f60993 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:30.546409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e518aa5-02cf-4c3b-b132-aa0cc0f60993 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:30.552738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:30.553823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:30.554200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:30.555103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] PATH_INFO: `/v3/users/2bc0aa5115cb455c9576eceaa655fb93` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:30.557154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc0aa5115cb455c9576eceaa655fb93)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:30.560200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:30.639713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc0aa5115cb455c9576eceaa655fb93'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:30.650347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:30.662216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc0aa5115cb455c9576eceaa655fb93'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:30.663293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc0aa5115cb455c9576eceaa655fb93'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:30.664126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc0aa5115cb455c9576eceaa655fb93'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:30.680390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc0aa5115cb455c9576eceaa655fb93'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:30.700381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1095/2175] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:30 2022] DELETE /identity/v3/users/2bc0aa5115cb455c9576eceaa655fb93 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:30.705058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b1791-cf01-4635-b408-7b20ce49058b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:30.705916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b1791-cf01-4635-b408-7b20ce49058b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:30.705916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b1791-cf01-4635-b408-7b20ce49058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:30.727252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-2e8b1791-cf01-4635-b408-7b20ce49058b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:30.729139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2e8b1791-cf01-4635-b408-7b20ce49058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:30.810060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1081/2176] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:30.831938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:30.834567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:30.916462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2552523-1433-40d3-98e1-101ee6ba6e9a admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:30.921187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890235f0-309b-47ff-ab09-84652e85c177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:30.922009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890235f0-309b-47ff-ab09-84652e85c177 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:30.922296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890235f0-309b-47ff-ab09-84652e85c177 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:30.922576 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890235f0-309b-47ff-ab09-84652e85c177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:30.938913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890235f0-309b-47ff-ab09-84652e85c177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:30.940785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890235f0-309b-47ff-ab09-84652e85c177 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:30.961495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1096/2177] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:50:31.840965 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b1791-cf01-4635-b408-7b20ce49058b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:31.842589 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2e8b1791-cf01-4635-b408-7b20ce49058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:31.860509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b1791-cf01-4635-b408-7b20ce49058b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:31.876770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2348b-c743-4baa-b359-da89e8bd14ac None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:31.877235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2348b-c743-4baa-b359-da89e8bd14ac None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:31.877504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d2348b-c743-4baa-b359-da89e8bd14ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:31.880040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:31.881289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:31.882117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:31.882408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] PATH_INFO: `/v3/projects/b3679689d28449bb9c8a5df37de4fb92` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:31.897943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3679689d28449bb9c8a5df37de4fb92)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:31.899925 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:31.922458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-93d2348b-c743-4baa-b359-da89e8bd14ac None None] MFA Rules not processed for user `7faba3540b1b4ef5a533ab0ff8e1cca3`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:31.923742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-93d2348b-c743-4baa-b359-da89e8bd14ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:31.991320 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3679689d28449bb9c8a5df37de4fb92', 'request_id': 'req-2e4a3db9-622b-47c7-81f0-fe172c308a13'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.003087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3679689d28449bb9c8a5df37de4fb92', 'request_id': 'req-2e4a3db9-622b-47c7-81f0-fe172c308a13'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.016816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3679689d28449bb9c8a5df37de4fb92', 'request_id': 'req-2e4a3db9-622b-47c7-81f0-fe172c308a13'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.032225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] The token cache is being invalidate because project b3679689d28449bb9c8a5df37de4fb92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:32.032808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.033688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.035421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1082/2178] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:31 2022] DELETE /identity/v3/projects/b3679689d28449bb9c8a5df37de4fb92 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:32.039829 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:32.042336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:32.048667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1097/2179] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:32.049679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2348b-c743-4baa-b359-da89e8bd14ac None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:32.053327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-93d2348b-c743-4baa-b359-da89e8bd14ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:32.068281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:32.079872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d2348b-c743-4baa-b359-da89e8bd14ac None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:32.155104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a3db9-622b-47c7-81f0-fe172c308a13 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:32.160435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:32.161303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.161590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.161900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] PATH_INFO: `/v3/users/3010644ebaac4fbab4996db86cb31a95` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.163068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3010644ebaac4fbab4996db86cb31a95)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:32.163508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472bdb1-9a47-4bcf-8627-d6bb139c2922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:32.164180 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472bdb1-9a47-4bcf-8627-d6bb139c2922 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.164469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472bdb1-9a47-4bcf-8627-d6bb139c2922 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.164779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472bdb1-9a47-4bcf-8627-d6bb139c2922 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.165483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472bdb1-9a47-4bcf-8627-d6bb139c2922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:32.165592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:32.168363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472bdb1-9a47-4bcf-8627-d6bb139c2922 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:32.178950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:32.203130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1098/2180] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:32 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:32.207072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472bdb1-9a47-4bcf-8627-d6bb139c2922 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:32.210078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6472bdb1-9a47-4bcf-8627-d6bb139c2922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:32.225869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3010644ebaac4fbab4996db86cb31a95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.246955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3010644ebaac4fbab4996db86cb31a95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.256539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3010644ebaac4fbab4996db86cb31a95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.257491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3010644ebaac4fbab4996db86cb31a95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.265719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3010644ebaac4fbab4996db86cb31a95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.272915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1083/2181] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:32 2022] DELETE /identity/v3/users/3010644ebaac4fbab4996db86cb31a95 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:32.287932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472bdb1-9a47-4bcf-8627-d6bb139c2922 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:32.349614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6994ffe-9de4-4a4d-adb9-716e31fed268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:32.350355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6994ffe-9de4-4a4d-adb9-716e31fed268 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.350642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6994ffe-9de4-4a4d-adb9-716e31fed268 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.350921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6994ffe-9de4-4a4d-adb9-716e31fed268 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.351660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6994ffe-9de4-4a4d-adb9-716e31fed268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:32.353871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6994ffe-9de4-4a4d-adb9-716e31fed268 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:32.377796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1099/2182] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:32 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:32.378963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:32.380483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:32.395262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05576940-f69d-4812-b103-95a4662c0230 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:32.401391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4b48e-4d96-42c0-8643-35f42f43c503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:32.402062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4b48e-4d96-42c0-8643-35f42f43c503 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.402410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4b48e-4d96-42c0-8643-35f42f43c503 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.402718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4b48e-4d96-42c0-8643-35f42f43c503 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.403477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4b48e-4d96-42c0-8643-35f42f43c503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:32.405099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4b48e-4d96-42c0-8643-35f42f43c503 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:32.416488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1084/2183] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:32.420758 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6994ffe-9de4-4a4d-adb9-716e31fed268 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:32.431070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6994ffe-9de4-4a4d-adb9-716e31fed268 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:32.436238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d3510-0944-4fdf-8471-179e076096fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:32.437366 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d3510-0944-4fdf-8471-179e076096fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.437578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d3510-0944-4fdf-8471-179e076096fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.437868 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d3510-0944-4fdf-8471-179e076096fc admin admin] PATH_INFO: `/v3/projects/edff0e7d198c4c93b8bb38caffc71217/users/e71329a343964a4d9a53ca144652da58/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.457696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d3510-0944-4fdf-8471-179e076096fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=edff0e7d198c4c93b8bb38caffc71217, user_id=e71329a343964a4d9a53ca144652da58, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:32.460383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d3510-0944-4fdf-8471-179e076096fc admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:32.476361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1100/2184] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:32 2022] PUT /identity/v3/projects/edff0e7d198c4c93b8bb38caffc71217/users/e71329a343964a4d9a53ca144652da58/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:32.481338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4b48e-4d96-42c0-8643-35f42f43c503 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:32.492540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4b48e-4d96-42c0-8643-35f42f43c503 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:32.568718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5266a8c-6c01-44bd-935f-b0cecb774f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:32.569542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5266a8c-6c01-44bd-935f-b0cecb774f94 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.569832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5266a8c-6c01-44bd-935f-b0cecb774f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.570119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5266a8c-6c01-44bd-935f-b0cecb774f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.570855 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5266a8c-6c01-44bd-935f-b0cecb774f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:32.572709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5266a8c-6c01-44bd-935f-b0cecb774f94 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:32.580388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d3510-0944-4fdf-8471-179e076096fc admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:32.583847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-cb7d3510-0944-4fdf-8471-179e076096fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:32.590626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1085/2185] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:32.596120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5266a8c-6c01-44bd-935f-b0cecb774f94 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:32.604344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d3510-0944-4fdf-8471-179e076096fc admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:32.611951 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5266a8c-6c01-44bd-935f-b0cecb774f94 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:32.615900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:32.617542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671599c-295f-485e-ab79-85e8461045e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:32.618446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671599c-295f-485e-ab79-85e8461045e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.618573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.618728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671599c-295f-485e-ab79-85e8461045e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.619009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671599c-295f-485e-ab79-85e8461045e6 admin admin] PATH_INFO: `/v3/projects/edff0e7d198c4c93b8bb38caffc71217/users/e71329a343964a4d9a53ca144652da58/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.620797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.621675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671599c-295f-485e-ab79-85e8461045e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edff0e7d198c4c93b8bb38caffc71217, user_id=e71329a343964a4d9a53ca144652da58, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:32.622070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] PATH_INFO: `/v3/projects/3f0fd62432f04569b068e432ac2d250e` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.624393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f0fd62432f04569b068e432ac2d250e)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:32.629011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:32.632848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671599c-295f-485e-ab79-85e8461045e6 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:32.641802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:32.659227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1086/2186] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:32 2022] PUT /identity/v3/projects/edff0e7d198c4c93b8bb38caffc71217/users/e71329a343964a4d9a53ca144652da58/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:32.710640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f0fd62432f04569b068e432ac2d250e', 'request_id': 'req-0d40c155-62cf-4cf5-941e-03b7fc178f32'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.730056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f0fd62432f04569b068e432ac2d250e', 'request_id': 'req-0d40c155-62cf-4cf5-941e-03b7fc178f32'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.732488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:32.738067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f0fd62432f04569b068e432ac2d250e', 'request_id': 'req-0d40c155-62cf-4cf5-941e-03b7fc178f32'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.752247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] The token cache is being invalidate because project 3f0fd62432f04569b068e432ac2d250e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:32.752546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.753393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0d40c155-62cf-4cf5-941e-03b7fc178f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:32.754922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1101/2187] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:32 2022] DELETE /identity/v3/projects/3f0fd62432f04569b068e432ac2d250e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:32.760047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb42a8-305a-431e-b51e-08147ecc5366 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.760342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb42a8-305a-431e-b51e-08147ecc5366 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.760657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb42a8-305a-431e-b51e-08147ecc5366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.781810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c8fb42a8-305a-431e-b51e-08147ecc5366 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:32.783054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c8fb42a8-305a-431e-b51e-08147ecc5366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:32.868571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1087/2188] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:32.879022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb0fd-3f71-4739-901c-7849688ab413 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.879460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb0fd-3f71-4739-901c-7849688ab413 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.879640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb0fd-3f71-4739-901c-7849688ab413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.905630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-5aebb0fd-3f71-4739-901c-7849688ab413 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:32.907721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5aebb0fd-3f71-4739-901c-7849688ab413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:32.917530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1102/2189] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:32.921896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb42a8-305a-431e-b51e-08147ecc5366 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:32.952884 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb42a8-305a-431e-b51e-08147ecc5366 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:32.965961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215d8c-8655-439e-b71d-cd371bd479c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:32.967271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215d8c-8655-439e-b71d-cd371bd479c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:32.967607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215d8c-8655-439e-b71d-cd371bd479c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:32.967890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215d8c-8655-439e-b71d-cd371bd479c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:32.968942 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215d8c-8655-439e-b71d-cd371bd479c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:32.969254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215d8c-8655-439e-b71d-cd371bd479c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:32.972025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215d8c-8655-439e-b71d-cd371bd479c7 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:32.985197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1088/2190] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:50:32.990205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb0fd-3f71-4739-901c-7849688ab413 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:33.000329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb0fd-3f71-4739-901c-7849688ab413 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:33.005374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5650c4-6c4f-467c-8ef5-0e3881fc8bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:33.005926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5650c4-6c4f-467c-8ef5-0e3881fc8bae admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:33.006274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5650c4-6c4f-467c-8ef5-0e3881fc8bae admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:33.006625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5650c4-6c4f-467c-8ef5-0e3881fc8bae admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:33.007404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5650c4-6c4f-467c-8ef5-0e3881fc8bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:33.009765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5650c4-6c4f-467c-8ef5-0e3881fc8bae admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:33.050242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1103/2191] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:32 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:33.061623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215d8c-8655-439e-b71d-cd371bd479c7 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:33.077078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215d8c-8655-439e-b71d-cd371bd479c7 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:33.184544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3155ce-5b56-4e97-9fa6-db1ad39f9b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:33.188250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3155ce-5b56-4e97-9fa6-db1ad39f9b5a admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:33.188720 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3155ce-5b56-4e97-9fa6-db1ad39f9b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:33.189407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3155ce-5b56-4e97-9fa6-db1ad39f9b5a admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:33.190256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3155ce-5b56-4e97-9fa6-db1ad39f9b5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:33.195674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3155ce-5b56-4e97-9fa6-db1ad39f9b5a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:33.248831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1089/2192] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:33 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:33.253050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5650c4-6c4f-467c-8ef5-0e3881fc8bae admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:33.266814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5650c4-6c4f-467c-8ef5-0e3881fc8bae admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:33.273457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49fca7-b990-4618-8ab2-021940ede578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:33.274217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49fca7-b990-4618-8ab2-021940ede578 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:33.274577 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49fca7-b990-4618-8ab2-021940ede578 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:33.274921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49fca7-b990-4618-8ab2-021940ede578 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:33.276737 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49fca7-b990-4618-8ab2-021940ede578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:33.278555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc49fca7-b990-4618-8ab2-021940ede578 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:33.294641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1104/2193] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:50:33.296267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3155ce-5b56-4e97-9fa6-db1ad39f9b5a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:33.308753 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3155ce-5b56-4e97-9fa6-db1ad39f9b5a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:33.320701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8f971-e673-4420-b786-c13676120299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:33.321407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f971-e673-4420-b786-c13676120299 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:33.321749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f971-e673-4420-b786-c13676120299 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:33.322033 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8f971-e673-4420-b786-c13676120299 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:33.323528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8f971-e673-4420-b786-c13676120299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:33.325504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8f971-e673-4420-b786-c13676120299 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:33.340685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1090/2194] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:33.344774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49fca7-b990-4618-8ab2-021940ede578 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:33.356229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49fca7-b990-4618-8ab2-021940ede578 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:33.361288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43bb14-10d1-4d24-b751-149e3e1bcab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:33.362244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43bb14-10d1-4d24-b751-149e3e1bcab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:33.362593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43bb14-10d1-4d24-b751-149e3e1bcab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:33.362936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43bb14-10d1-4d24-b751-149e3e1bcab2 admin admin] PATH_INFO: `/v3/projects/f5f0c98def5942cea5329ba1ec81c4d9/users/305561441f614d55829c3f6182db360f/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:33.380253 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43bb14-10d1-4d24-b751-149e3e1bcab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f0c98def5942cea5329ba1ec81c4d9, user_id=305561441f614d55829c3f6182db360f, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:33.382987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43bb14-10d1-4d24-b751-149e3e1bcab2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:33.409482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1105/2195] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:33 2022] PUT /identity/v3/projects/f5f0c98def5942cea5329ba1ec81c4d9/users/305561441f614d55829c3f6182db360f/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:33.415468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a389f2-e912-4c8f-982e-b1f4124752e8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:33.415793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a389f2-e912-4c8f-982e-b1f4124752e8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:33.416335 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a389f2-e912-4c8f-982e-b1f4124752e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:33.453024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-49a389f2-e912-4c8f-982e-b1f4124752e8 None None] MFA Rules not processed for user `305561441f614d55829c3f6182db360f`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:33.454870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-49a389f2-e912-4c8f-982e-b1f4124752e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:33.555939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:33.588255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1091/2196] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:33.599071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43bb14-10d1-4d24-b751-149e3e1bcab2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:33.601631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-fa43bb14-10d1-4d24-b751-149e3e1bcab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:33.688707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43bb14-10d1-4d24-b751-149e3e1bcab2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:33.694224 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24334c-02c8-44d2-b751-bde679569265 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71e5ef9dc32f494791ba4addb5151232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:33.695394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24334c-02c8-44d2-b751-bde679569265 service glance] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:33.695394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24334c-02c8-44d2-b751-bde679569265 service glance] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:33.695689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24334c-02c8-44d2-b751-bde679569265 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:33.708532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24334c-02c8-44d2-b751-bde679569265 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:33.710447 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24334c-02c8-44d2-b751-bde679569265 service glance] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:33.724795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1106/2197] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:50:39.089751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb19e1c-6cb9-415a-8bfd-d3f023eb72a5 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:39.090256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb19e1c-6cb9-415a-8bfd-d3f023eb72a5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:39.090350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb19e1c-6cb9-415a-8bfd-d3f023eb72a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:39.125283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-dfb19e1c-6cb9-415a-8bfd-d3f023eb72a5 None None] MFA Rules not processed for user `e71329a343964a4d9a53ca144652da58`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:39.126443 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-dfb19e1c-6cb9-415a-8bfd-d3f023eb72a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:39.201974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1092/2198] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:39 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:39.217169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24334c-02c8-44d2-b751-bde679569265 service glance] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:39.218585 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-fa24334c-02c8-44d2-b751-bde679569265 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:39.305093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24334c-02c8-44d2-b751-bde679569265 service glance] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:39.310831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64758a94-f025-48b7-bb38-7c09619a580b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:39.311496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758a94-f025-48b7-bb38-7c09619a580b service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:39.311783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758a94-f025-48b7-bb38-7c09619a580b service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:39.312066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758a94-f025-48b7-bb38-7c09619a580b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:39.313847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-64758a94-f025-48b7-bb38-7c09619a580b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:39.339298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:39.397227 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64758a94-f025-48b7-bb38-7c09619a580b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:39.397227 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64758a94-f025-48b7-bb38-7c09619a580b service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:39.405355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1107/2199] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:40.583090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43da6b-f176-4f95-a7e2-13f11d22a61a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:40.583625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43da6b-f176-4f95-a7e2-13f11d22a61a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:40.584043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43da6b-f176-4f95-a7e2-13f11d22a61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:40.607176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8e43da6b-f176-4f95-a7e2-13f11d22a61a None None] MFA Rules not processed for user `7faba3540b1b4ef5a533ab0ff8e1cca3`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:40.608429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8e43da6b-f176-4f95-a7e2-13f11d22a61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:40.615563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1093/2200] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:43.293371 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcad47-17ce-4551-893d-297c759ffd15 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:43.293891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcad47-17ce-4551-893d-297c759ffd15 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:43.294414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcad47-17ce-4551-893d-297c759ffd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:43.312013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b2fcad47-17ce-4551-893d-297c759ffd15 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:43.313312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b2fcad47-17ce-4551-893d-297c759ffd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:43.387348 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1108/2201] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:50:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:43.432390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43da6b-f176-4f95-a7e2-13f11d22a61a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:43.433968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8e43da6b-f176-4f95-a7e2-13f11d22a61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:43.537027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43da6b-f176-4f95-a7e2-13f11d22a61a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:43.543533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222621d-1719-49d3-aedf-069c0d97a56b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:43.544211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222621d-1719-49d3-aedf-069c0d97a56b service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:43.544501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222621d-1719-49d3-aedf-069c0d97a56b service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:43.544855 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222621d-1719-49d3-aedf-069c0d97a56b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:43.559079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222621d-1719-49d3-aedf-069c0d97a56b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:43.560913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222621d-1719-49d3-aedf-069c0d97a56b service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:43.570875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1094/2202] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:50:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:50:44.546412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161021a6-2f11-41b1-a30a-702a2bd0ec0b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:44.547380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161021a6-2f11-41b1-a30a-702a2bd0ec0b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:44.547660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161021a6-2f11-41b1-a30a-702a2bd0ec0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:44.567706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-161021a6-2f11-41b1-a30a-702a2bd0ec0b None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:44.568956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-161021a6-2f11-41b1-a30a-702a2bd0ec0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:44.577306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1109/2203] 10.222.0.47 () {58 vars in 967 bytes} [Fri Jun 24 23:50:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:44.593412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222621d-1719-49d3-aedf-069c0d97a56b service swift] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:44.608003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222621d-1719-49d3-aedf-069c0d97a56b service swift] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:44.615899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0cc68-fd7d-4100-8839-09c53fda67ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:44.616572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0cc68-fd7d-4100-8839-09c53fda67ad service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:44.616882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0cc68-fd7d-4100-8839-09c53fda67ad service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:44.617156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0cc68-fd7d-4100-8839-09c53fda67ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:44.627507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0cc68-fd7d-4100-8839-09c53fda67ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:44.629271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0cc68-fd7d-4100-8839-09c53fda67ad service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:44.641796 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1095/2204] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:50:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:50:44.867551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710d4eb-5aa2-4dba-b1f8-dbd428aa6ae2 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:44.867842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710d4eb-5aa2-4dba-b1f8-dbd428aa6ae2 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:44.868122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710d4eb-5aa2-4dba-b1f8-dbd428aa6ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:44.890167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-4710d4eb-5aa2-4dba-b1f8-dbd428aa6ae2 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:44.891484 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4710d4eb-5aa2-4dba-b1f8-dbd428aa6ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:44.899179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1110/2205] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:50:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:44.911545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0cc68-fd7d-4100-8839-09c53fda67ad service swift] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:44.926744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0cc68-fd7d-4100-8839-09c53fda67ad service swift] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:44.934614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b160f-ecd6-4126-81fc-e689378d3026 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:44.935283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b160f-ecd6-4126-81fc-e689378d3026 service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:44.935570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b160f-ecd6-4126-81fc-e689378d3026 service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:44.935848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b160f-ecd6-4126-81fc-e689378d3026 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:44.948510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b160f-ecd6-4126-81fc-e689378d3026 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:44.950144 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b160f-ecd6-4126-81fc-e689378d3026 service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:44.959034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1096/2206] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:50:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:50:45.333213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cfbe41-9fdf-4f10-ae47-f5aefc9e572c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:45.333500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cfbe41-9fdf-4f10-ae47-f5aefc9e572c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:45.333952 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cfbe41-9fdf-4f10-ae47-f5aefc9e572c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:45.352718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-46cfbe41-9fdf-4f10-ae47-f5aefc9e572c None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:45.354157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-46cfbe41-9fdf-4f10-ae47-f5aefc9e572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:45.361286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1111/2207] 10.222.0.47 () {58 vars in 967 bytes} [Fri Jun 24 23:50:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:45.374931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b160f-ecd6-4126-81fc-e689378d3026 service swift] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:45.386474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b160f-ecd6-4126-81fc-e689378d3026 service swift] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:45.391648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d6380-c2b7-4187-bdb8-ae6f978bad19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:45.392290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d6380-c2b7-4187-bdb8-ae6f978bad19 service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:45.392702 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d6380-c2b7-4187-bdb8-ae6f978bad19 service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:45.392990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d6380-c2b7-4187-bdb8-ae6f978bad19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:45.402654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d6380-c2b7-4187-bdb8-ae6f978bad19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:45.404227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d6380-c2b7-4187-bdb8-ae6f978bad19 service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:45.412464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1097/2208] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:50:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 23:50:47.572693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cfbe41-9fdf-4f10-ae47-f5aefc9e572c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:47.574450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-46cfbe41-9fdf-4f10-ae47-f5aefc9e572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:47.641876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cfbe41-9fdf-4f10-ae47-f5aefc9e572c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:47.646464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:47.647589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:47.647870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:47.648143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] PATH_INFO: `/v3/users/ac07e1b2de6f4607a37047b6edee4704` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:47.649385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac07e1b2de6f4607a37047b6edee4704)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:47.651894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:47.701823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac07e1b2de6f4607a37047b6edee4704'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:47.704148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:47.724274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac07e1b2de6f4607a37047b6edee4704'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:47.724862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac07e1b2de6f4607a37047b6edee4704'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:47.725475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac07e1b2de6f4607a37047b6edee4704'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:47.734668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac07e1b2de6f4607a37047b6edee4704'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:47.748504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1112/2209] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:47 2022] DELETE /identity/v3/users/ac07e1b2de6f4607a37047b6edee4704 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:47.754846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4c498-e748-4cec-8d8f-6444e0fafc8b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:47.755398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4c498-e748-4cec-8d8f-6444e0fafc8b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:47.755398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4c498-e748-4cec-8d8f-6444e0fafc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:47.774267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-87d4c498-e748-4cec-8d8f-6444e0fafc8b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:47.775806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-87d4c498-e748-4cec-8d8f-6444e0fafc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:47.793969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:47.839915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1098/2210] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:47.849588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:47.853069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:47.913935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d39d6-685c-4409-a677-3c971fcd2189 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:47.918445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9c026-4396-4264-bf75-38ea9d17cbbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:47.919038 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9c026-4396-4264-bf75-38ea9d17cbbf service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:47.919323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9c026-4396-4264-bf75-38ea9d17cbbf service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:47.919601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9c026-4396-4264-bf75-38ea9d17cbbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:47.929032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9c026-4396-4264-bf75-38ea9d17cbbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:47.930465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9c026-4396-4264-bf75-38ea9d17cbbf service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:47.942283 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1113/2211] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:48.136307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4c498-e748-4cec-8d8f-6444e0fafc8b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:48.137804 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-87d4c498-e748-4cec-8d8f-6444e0fafc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:48.152493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4c498-e748-4cec-8d8f-6444e0fafc8b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:48.157580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:48.158547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.158825 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.159114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] PATH_INFO: `/v3/projects/07b87adb66004fb39c8558287d1bee98` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.160326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b87adb66004fb39c8558287d1bee98)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:48.162360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:48.212803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b87adb66004fb39c8558287d1bee98', 'request_id': 'req-05e885a0-5ded-417a-92d7-e507767cd3db'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:48.224222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b87adb66004fb39c8558287d1bee98', 'request_id': 'req-05e885a0-5ded-417a-92d7-e507767cd3db'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:48.237469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b87adb66004fb39c8558287d1bee98', 'request_id': 'req-05e885a0-5ded-417a-92d7-e507767cd3db'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:48.251325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] The token cache is being invalidate because project 07b87adb66004fb39c8558287d1bee98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:48.251666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:48.252336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-05e885a0-5ded-417a-92d7-e507767cd3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:48.254172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1099/2212] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:48 2022] DELETE /identity/v3/projects/07b87adb66004fb39c8558287d1bee98 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:50:48.260977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff3646-592c-4682-9a85-09c34de4f754 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.261392 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff3646-592c-4682-9a85-09c34de4f754 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.261673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff3646-592c-4682-9a85-09c34de4f754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.279844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b2ff3646-592c-4682-9a85-09c34de4f754 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:48.281115 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b2ff3646-592c-4682-9a85-09c34de4f754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:48.345257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1114/2213] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:48.354441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ae402-10c7-4e10-b263-06096ae55bb9 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.354734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ae402-10c7-4e10-b263-06096ae55bb9 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.355022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ae402-10c7-4e10-b263-06096ae55bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.374968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-dd7ae402-10c7-4e10-b263-06096ae55bb9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:48.376175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-dd7ae402-10c7-4e10-b263-06096ae55bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:48.383988 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1100/2214] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:48.389152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff3646-592c-4682-9a85-09c34de4f754 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:48.399483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff3646-592c-4682-9a85-09c34de4f754 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:48.404990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0524c05-50a9-4821-8e50-65bd27eb226d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:48.405713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0524c05-50a9-4821-8e50-65bd27eb226d admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.405998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0524c05-50a9-4821-8e50-65bd27eb226d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.406276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0524c05-50a9-4821-8e50-65bd27eb226d admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.407012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0524c05-50a9-4821-8e50-65bd27eb226d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:50:48.407331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0524c05-50a9-4821-8e50-65bd27eb226d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:48.408969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0524c05-50a9-4821-8e50-65bd27eb226d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:48.421270 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1115/2215] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:50:48.425675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ae402-10c7-4e10-b263-06096ae55bb9 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:48.435476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ae402-10c7-4e10-b263-06096ae55bb9 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:48.440168 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbee32-0e5b-452e-afc8-fc00c4ddce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:48.440852 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbee32-0e5b-452e-afc8-fc00c4ddce5a admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.441134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbee32-0e5b-452e-afc8-fc00c4ddce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.441410 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbee32-0e5b-452e-afc8-fc00c4ddce5a admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.442114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbee32-0e5b-452e-afc8-fc00c4ddce5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:48.443987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbee32-0e5b-452e-afc8-fc00c4ddce5a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:48.473986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1101/2216] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:50:48 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:50:48.478851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0524c05-50a9-4821-8e50-65bd27eb226d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:48.489013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0524c05-50a9-4821-8e50-65bd27eb226d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:48.542152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:48.549023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53cf8e-f2cc-4906-bfc6-d59f5a453a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:48.549728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53cf8e-f2cc-4906-bfc6-d59f5a453a93 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.550147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53cf8e-f2cc-4906-bfc6-d59f5a453a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.550789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53cf8e-f2cc-4906-bfc6-d59f5a453a93 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.551586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53cf8e-f2cc-4906-bfc6-d59f5a453a93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:48.553647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53cf8e-f2cc-4906-bfc6-d59f5a453a93 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:48.573996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1116/2217] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:50:48 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:50:48.578580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbee32-0e5b-452e-afc8-fc00c4ddce5a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:48.591978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbee32-0e5b-452e-afc8-fc00c4ddce5a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:48.597286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b07c8-40f0-4d07-902a-d590522e7e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:48.597986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b07c8-40f0-4d07-902a-d590522e7e7e admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.598299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b07c8-40f0-4d07-902a-d590522e7e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.598578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b07c8-40f0-4d07-902a-d590522e7e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.599328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b07c8-40f0-4d07-902a-d590522e7e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:48.600961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b07c8-40f0-4d07-902a-d590522e7e7e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:48.612546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1102/2218] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:48.616474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53cf8e-f2cc-4906-bfc6-d59f5a453a93 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:48.626966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53cf8e-f2cc-4906-bfc6-d59f5a453a93 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:48.631814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6cb78-f4a2-4438-bcbc-c323a3efd940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:48.632713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6cb78-f4a2-4438-bcbc-c323a3efd940 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.633007 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6cb78-f4a2-4438-bcbc-c323a3efd940 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.633362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6cb78-f4a2-4438-bcbc-c323a3efd940 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.634200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6cb78-f4a2-4438-bcbc-c323a3efd940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:48.636299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6cb78-f4a2-4438-bcbc-c323a3efd940 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:48.647021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1117/2219] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:50:48.651745 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b07c8-40f0-4d07-902a-d590522e7e7e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:48.661550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b07c8-40f0-4d07-902a-d590522e7e7e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:48.666699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98c864-bf42-4b0a-9bb3-384791ffaf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:48.667569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c864-bf42-4b0a-9bb3-384791ffaf39 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.667844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c864-bf42-4b0a-9bb3-384791ffaf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.668116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98c864-bf42-4b0a-9bb3-384791ffaf39 admin admin] PATH_INFO: `/v3/projects/ca610df7feca4e1b90ca8dca11bab2e0/users/d1af895ac2364bffbadadfb41dd3a5f5/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.683238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98c864-bf42-4b0a-9bb3-384791ffaf39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca610df7feca4e1b90ca8dca11bab2e0, user_id=d1af895ac2364bffbadadfb41dd3a5f5, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:48.685947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98c864-bf42-4b0a-9bb3-384791ffaf39 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:48.702628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1103/2220] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:50:48 2022] PUT /identity/v3/projects/ca610df7feca4e1b90ca8dca11bab2e0/users/d1af895ac2364bffbadadfb41dd3a5f5/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:48.771927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daad9c-67b4-4db9-9439-a05654ded930 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.772220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daad9c-67b4-4db9-9439-a05654ded930 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.772515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daad9c-67b4-4db9-9439-a05654ded930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.802278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b0daad9c-67b4-4db9-9439-a05654ded930 None None] MFA Rules not processed for user `d1af895ac2364bffbadadfb41dd3a5f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:48.803914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b0daad9c-67b4-4db9-9439-a05654ded930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:48.869649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1118/2221] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:50:48.882270 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98c864-bf42-4b0a-9bb3-384791ffaf39 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:48.884142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7a98c864-bf42-4b0a-9bb3-384791ffaf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:48.930374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:48.944864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98c864-bf42-4b0a-9bb3-384791ffaf39 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:48.949626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea589027-a6c7-4a80-a0d7-173887e9a707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:48.950285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea589027-a6c7-4a80-a0d7-173887e9a707 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:48.950564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea589027-a6c7-4a80-a0d7-173887e9a707 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:48.950839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea589027-a6c7-4a80-a0d7-173887e9a707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:48.961947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea589027-a6c7-4a80-a0d7-173887e9a707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:48.964162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea589027-a6c7-4a80-a0d7-173887e9a707 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:48.974722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1104/2222] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:57.320488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0daad9c-67b4-4db9-9439-a05654ded930 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:57.407037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0daad9c-67b4-4db9-9439-a05654ded930 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:57.412238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:57.413872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:57.414208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:57.414485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] PATH_INFO: `/v3/users/d1af895ac2364bffbadadfb41dd3a5f5` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:57.415810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1af895ac2364bffbadadfb41dd3a5f5)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:57.418123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:57.468221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1af895ac2364bffbadadfb41dd3a5f5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:57.483164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1af895ac2364bffbadadfb41dd3a5f5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:57.484052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1af895ac2364bffbadadfb41dd3a5f5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:57.484677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1af895ac2364bffbadadfb41dd3a5f5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:57.487973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:50:57.501115 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1af895ac2364bffbadadfb41dd3a5f5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:57.516835 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1119/2223] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:50:57 2022] DELETE /identity/v3/users/d1af895ac2364bffbadadfb41dd3a5f5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:50:57.521749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fdf3b-7bba-43ef-9085-d9cbcbc2fc12 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:57.522850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fdf3b-7bba-43ef-9085-d9cbcbc2fc12 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:57.523360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0fdf3b-7bba-43ef-9085-d9cbcbc2fc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:57.542348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-2b0fdf3b-7bba-43ef-9085-d9cbcbc2fc12 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:50:57.543544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2b0fdf3b-7bba-43ef-9085-d9cbcbc2fc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:57.610110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1105/2224] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:50:57.619549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:57.620979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:57.682107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af60c7-42df-4348-86f0-432e75dc3f6f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:57.686793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ffc58-64cf-47fc-b187-c5ee8b690bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:57.687438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ffc58-64cf-47fc-b187-c5ee8b690bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:57.687721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ffc58-64cf-47fc-b187-c5ee8b690bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:57.687999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ffc58-64cf-47fc-b187-c5ee8b690bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:57.697790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ffc58-64cf-47fc-b187-c5ee8b690bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:57.699267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ffc58-64cf-47fc-b187-c5ee8b690bd3 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:57.711658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1120/2225] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:50:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:50:57.982525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fdf3b-7bba-43ef-9085-d9cbcbc2fc12 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:50:57.983950 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2b0fdf3b-7bba-43ef-9085-d9cbcbc2fc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:50:57.996004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0fdf3b-7bba-43ef-9085-d9cbcbc2fc12 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:50:58.005789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:50:58.006682 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:50:58.006963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:50:58.007239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] PATH_INFO: `/v3/projects/ca610df7feca4e1b90ca8dca11bab2e0` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:50:58.008509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca610df7feca4e1b90ca8dca11bab2e0)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:50:58.016646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:50:58.068703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca610df7feca4e1b90ca8dca11bab2e0', 'request_id': 'req-9a5d132e-16af-41d7-9e94-118e163e0b2c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:58.075976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca610df7feca4e1b90ca8dca11bab2e0', 'request_id': 'req-9a5d132e-16af-41d7-9e94-118e163e0b2c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:58.086223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca610df7feca4e1b90ca8dca11bab2e0', 'request_id': 'req-9a5d132e-16af-41d7-9e94-118e163e0b2c'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:58.097453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] The token cache is being invalidate because project ca610df7feca4e1b90ca8dca11bab2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:50:58.097743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:58.098391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9a5d132e-16af-41d7-9e94-118e163e0b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:50:58.099931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1106/2226] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:50:57 2022] DELETE /identity/v3/projects/ca610df7feca4e1b90ca8dca11bab2e0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:07.706758 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:DELETE=1,keystone:SELECT=32 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:08.094351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:10.277321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ffc58-64cf-47fc-b187-c5ee8b690bd3 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:10.278789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-715ffc58-64cf-47fc-b187-c5ee8b690bd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:10.372286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ffc58-64cf-47fc-b187-c5ee8b690bd3 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:10.433195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:10.434016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:10.434292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:10.434582 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] PATH_INFO: `/v3/users/305561441f614d55829c3f6182db360f` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:10.435885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=305561441f614d55829c3f6182db360f)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:10.438390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:10.500519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305561441f614d55829c3f6182db360f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:10.516988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305561441f614d55829c3f6182db360f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:10.517971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305561441f614d55829c3f6182db360f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:10.518711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305561441f614d55829c3f6182db360f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:10.526816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305561441f614d55829c3f6182db360f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:10.540050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1121/2227] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:51:10 2022] DELETE /identity/v3/users/305561441f614d55829c3f6182db360f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:10.544623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef175e00-fe26-4604-805e-a5caebbaa396 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:10.545884 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef175e00-fe26-4604-805e-a5caebbaa396 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:10.546300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef175e00-fe26-4604-805e-a5caebbaa396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:10.576962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-ef175e00-fe26-4604-805e-a5caebbaa396 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:10.578211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ef175e00-fe26-4604-805e-a5caebbaa396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:10.645892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1107/2228] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:10.656189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:10.657640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:10.683299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:10.757927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d9207-d98f-4e0c-9209-4f53636b0bb0 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:10.761597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8090659-6895-43dd-b8ee-9749631e8bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:10.762243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090659-6895-43dd-b8ee-9749631e8bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:10.762523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090659-6895-43dd-b8ee-9749631e8bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:10.762797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090659-6895-43dd-b8ee-9749631e8bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:10.772267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8090659-6895-43dd-b8ee-9749631e8bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:10.773736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8090659-6895-43dd-b8ee-9749631e8bf0 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:10.783195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1122/2229] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:11.163148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef175e00-fe26-4604-805e-a5caebbaa396 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:11.164792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ef175e00-fe26-4604-805e-a5caebbaa396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:11.177611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef175e00-fe26-4604-805e-a5caebbaa396 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:11.182323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:11.183185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.183465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.183793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] PATH_INFO: `/v3/projects/f5f0c98def5942cea5329ba1ec81c4d9` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.184978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f0c98def5942cea5329ba1ec81c4d9)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:11.186978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:11.236814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f0c98def5942cea5329ba1ec81c4d9', 'request_id': 'req-f96544cb-a110-4213-ac98-426e06ab3105'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:11.244692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f0c98def5942cea5329ba1ec81c4d9', 'request_id': 'req-f96544cb-a110-4213-ac98-426e06ab3105'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:11.254074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f0c98def5942cea5329ba1ec81c4d9', 'request_id': 'req-f96544cb-a110-4213-ac98-426e06ab3105'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:11.266203 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] The token cache is being invalidate because project f5f0c98def5942cea5329ba1ec81c4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:51:11.266639 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:11.267508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f96544cb-a110-4213-ac98-426e06ab3105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:11.269274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1108/2230] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:51:11 2022] DELETE /identity/v3/projects/f5f0c98def5942cea5329ba1ec81c4d9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:11.274847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3e92f-a715-4233-83ff-3ab2593050cc None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.275134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3e92f-a715-4233-83ff-3ab2593050cc None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.275433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3e92f-a715-4233-83ff-3ab2593050cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.313877 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-34a3e92f-a715-4233-83ff-3ab2593050cc None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:11.315109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-34a3e92f-a715-4233-83ff-3ab2593050cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:11.375059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1123/2231] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:11.382518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70514b-ec93-4c05-a3d8-339cb6e66bdb None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.382825 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70514b-ec93-4c05-a3d8-339cb6e66bdb None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.383087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70514b-ec93-4c05-a3d8-339cb6e66bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.401995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-aa70514b-ec93-4c05-a3d8-339cb6e66bdb None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:11.403185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-aa70514b-ec93-4c05-a3d8-339cb6e66bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:11.411088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1109/2232] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:11.415353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3e92f-a715-4233-83ff-3ab2593050cc None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:11.425777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3e92f-a715-4233-83ff-3ab2593050cc None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:11.430366 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea069244-d76b-48cb-ae64-e967d071347e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:11.431018 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea069244-d76b-48cb-ae64-e967d071347e admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.431306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea069244-d76b-48cb-ae64-e967d071347e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.431587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea069244-d76b-48cb-ae64-e967d071347e admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.432311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea069244-d76b-48cb-ae64-e967d071347e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:51:11.432655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea069244-d76b-48cb-ae64-e967d071347e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:11.434334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea069244-d76b-48cb-ae64-e967d071347e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:11.445825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1124/2233] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:51:11.449961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70514b-ec93-4c05-a3d8-339cb6e66bdb None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:11.459612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70514b-ec93-4c05-a3d8-339cb6e66bdb None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:11.464210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75749c3-2d16-4b8f-9e6b-8a7c7292d85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:11.464864 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75749c3-2d16-4b8f-9e6b-8a7c7292d85c admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.465148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75749c3-2d16-4b8f-9e6b-8a7c7292d85c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.465424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75749c3-2d16-4b8f-9e6b-8a7c7292d85c admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.466116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75749c3-2d16-4b8f-9e6b-8a7c7292d85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:11.468731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75749c3-2d16-4b8f-9e6b-8a7c7292d85c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:11.495671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1110/2234] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:51:11 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:51:11.500189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea069244-d76b-48cb-ae64-e967d071347e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:11.510570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea069244-d76b-48cb-ae64-e967d071347e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:11.568607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:11.575001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf87a1-e11f-42e7-93ec-ff9fd4b91992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:11.575703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf87a1-e11f-42e7-93ec-ff9fd4b91992 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.576558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf87a1-e11f-42e7-93ec-ff9fd4b91992 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.576858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf87a1-e11f-42e7-93ec-ff9fd4b91992 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.577667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf87a1-e11f-42e7-93ec-ff9fd4b91992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:11.579705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf87a1-e11f-42e7-93ec-ff9fd4b91992 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:11.599458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1125/2235] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:51:11 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:51:11.603735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75749c3-2d16-4b8f-9e6b-8a7c7292d85c admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:11.614768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75749c3-2d16-4b8f-9e6b-8a7c7292d85c admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:11.619460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c808a-ea48-48cc-a9e7-da63dd182ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:11.620112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c808a-ea48-48cc-a9e7-da63dd182ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.620424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c808a-ea48-48cc-a9e7-da63dd182ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.620741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c808a-ea48-48cc-a9e7-da63dd182ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.621476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c808a-ea48-48cc-a9e7-da63dd182ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:11.624135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c808a-ea48-48cc-a9e7-da63dd182ef8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:11.634568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1111/2236] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:51:11.639216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf87a1-e11f-42e7-93ec-ff9fd4b91992 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:11.650042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf87a1-e11f-42e7-93ec-ff9fd4b91992 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:11.655863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8ac1-1120-4311-bb0c-b11adf75b340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:11.656723 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8ac1-1120-4311-bb0c-b11adf75b340 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.657030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8ac1-1120-4311-bb0c-b11adf75b340 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.657310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8ac1-1120-4311-bb0c-b11adf75b340 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.658045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da8ac1-1120-4311-bb0c-b11adf75b340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:11.659653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da8ac1-1120-4311-bb0c-b11adf75b340 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:11.670691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1126/2237] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:51:11.674886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c808a-ea48-48cc-a9e7-da63dd182ef8 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:11.685029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c808a-ea48-48cc-a9e7-da63dd182ef8 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:11.690572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4fe54-9959-401a-be0e-4304444b1893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:11.691452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4fe54-9959-401a-be0e-4304444b1893 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.691733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4fe54-9959-401a-be0e-4304444b1893 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.692020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4fe54-9959-401a-be0e-4304444b1893 admin admin] PATH_INFO: `/v3/projects/486b8d2d5d394aec8db481b9874bb773/users/3809233794c14027ac6431ebd5cd1603/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.711686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4fe54-9959-401a-be0e-4304444b1893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486b8d2d5d394aec8db481b9874bb773, user_id=3809233794c14027ac6431ebd5cd1603, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:11.714772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4fe54-9959-401a-be0e-4304444b1893 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:11.727345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1112/2238] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:51:11 2022] PUT /identity/v3/projects/486b8d2d5d394aec8db481b9874bb773/users/3809233794c14027ac6431ebd5cd1603/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:51:11.738092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c962ea-61fd-4222-8ea0-8c0459b9ab46 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.738383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c962ea-61fd-4222-8ea0-8c0459b9ab46 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.738663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c962ea-61fd-4222-8ea0-8c0459b9ab46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.765907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-34c962ea-61fd-4222-8ea0-8c0459b9ab46 None None] MFA Rules not processed for user `3809233794c14027ac6431ebd5cd1603`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:11.767110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-34c962ea-61fd-4222-8ea0-8c0459b9ab46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:11.829584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1127/2239] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:11.840209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4fe54-9959-401a-be0e-4304444b1893 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:11.842213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-fbf4fe54-9959-401a-be0e-4304444b1893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:11.856134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:11.902447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4fe54-9959-401a-be0e-4304444b1893 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:11.906903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead51413-9263-4488-897b-002d61519b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:11.907579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead51413-9263-4488-897b-002d61519b5a service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:11.907855 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead51413-9263-4488-897b-002d61519b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:11.908135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead51413-9263-4488-897b-002d61519b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:11.917285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead51413-9263-4488-897b-002d61519b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:11.918716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead51413-9263-4488-897b-002d61519b5a service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:11.928094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1113/2240] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:16.534422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969acaf8-5102-45e0-9110-38b67dbb67ca None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:16.534968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969acaf8-5102-45e0-9110-38b67dbb67ca None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:16.534968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969acaf8-5102-45e0-9110-38b67dbb67ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:16.553295 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-969acaf8-5102-45e0-9110-38b67dbb67ca None None] MFA Rules not processed for user `5eb7e8b4befd4d519b440c632b59e724`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:16.554539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-969acaf8-5102-45e0-9110-38b67dbb67ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:16.615363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1128/2241] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:21.925068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=20 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:22.625405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead51413-9263-4488-897b-002d61519b5a service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:22.702535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead51413-9263-4488-897b-002d61519b5a service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:22.707776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:22.708915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:22.709203 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:22.710499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] PATH_INFO: `/v3/users/3809233794c14027ac6431ebd5cd1603` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:22.710768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3809233794c14027ac6431ebd5cd1603)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:22.713285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:22.799800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3809233794c14027ac6431ebd5cd1603'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:22.815751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3809233794c14027ac6431ebd5cd1603'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:22.827424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3809233794c14027ac6431ebd5cd1603'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:22.828249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3809233794c14027ac6431ebd5cd1603'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:22.839787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3809233794c14027ac6431ebd5cd1603'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:22.849607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1114/2242] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:51:22 2022] DELETE /identity/v3/users/3809233794c14027ac6431ebd5cd1603 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:22.854081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a736f4-7ac6-4303-8dc9-8257575b9413 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:22.854580 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a736f4-7ac6-4303-8dc9-8257575b9413 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:22.854663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a736f4-7ac6-4303-8dc9-8257575b9413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:22.879157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d9a736f4-7ac6-4303-8dc9-8257575b9413 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:22.880547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d9a736f4-7ac6-4303-8dc9-8257575b9413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:22.984856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1129/2243] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:23.000013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:23.001676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:23.107594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77b2b4-4243-4fe2-9274-5c669f346e87 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:23.116511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d42e4-f796-414f-b9af-aa2b807668a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:23.118708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d42e4-f796-414f-b9af-aa2b807668a8 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:23.119538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d42e4-f796-414f-b9af-aa2b807668a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:23.119850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d42e4-f796-414f-b9af-aa2b807668a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:23.177495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d42e4-f796-414f-b9af-aa2b807668a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:23.177495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d42e4-f796-414f-b9af-aa2b807668a8 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:23.213447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1115/2244] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:23.801459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba17b5-4c31-45e8-94c1-1de3db9ad761 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:23.801926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba17b5-4c31-45e8-94c1-1de3db9ad761 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:23.802532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba17b5-4c31-45e8-94c1-1de3db9ad761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:23.821560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:23.849338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f4ba17b5-4c31-45e8-94c1-1de3db9ad761 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:23.849338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f4ba17b5-4c31-45e8-94c1-1de3db9ad761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:23.949093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1130/2245] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:51:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:23.957229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d42e4-f796-414f-b9af-aa2b807668a8 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:23.959511 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3d1d42e4-f796-414f-b9af-aa2b807668a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:24.023049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:24.149279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d42e4-f796-414f-b9af-aa2b807668a8 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:24.164718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a816f26-e7eb-42f1-a142-e8c132a4ae63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:24.165405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a816f26-e7eb-42f1-a142-e8c132a4ae63 service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:24.165691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a816f26-e7eb-42f1-a142-e8c132a4ae63 service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:24.166102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a816f26-e7eb-42f1-a142-e8c132a4ae63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:24.182559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a816f26-e7eb-42f1-a142-e8c132a4ae63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:24.184188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a816f26-e7eb-42f1-a142-e8c132a4ae63 service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:24.222192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1116/2246] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:51:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 23:51:24.728271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba17b5-4c31-45e8-94c1-1de3db9ad761 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:24.730762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f4ba17b5-4c31-45e8-94c1-1de3db9ad761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:24.789830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba17b5-4c31-45e8-94c1-1de3db9ad761 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:24.821074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:24.823972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:24.824770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:24.826612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] PATH_INFO: `/v3/projects/486b8d2d5d394aec8db481b9874bb773` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:24.832235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=486b8d2d5d394aec8db481b9874bb773)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:24.836964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:24.922750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486b8d2d5d394aec8db481b9874bb773', 'request_id': 'req-31c17170-be68-4625-9a37-4026e45929ff'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:24.943293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486b8d2d5d394aec8db481b9874bb773', 'request_id': 'req-31c17170-be68-4625-9a37-4026e45929ff'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:24.953830 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486b8d2d5d394aec8db481b9874bb773', 'request_id': 'req-31c17170-be68-4625-9a37-4026e45929ff'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:24.976026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] The token cache is being invalidate because project 486b8d2d5d394aec8db481b9874bb773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:51:24.978246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:24.979970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-31c17170-be68-4625-9a37-4026e45929ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:24.991824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1131/2247] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:51:24 2022] DELETE /identity/v3/projects/486b8d2d5d394aec8db481b9874bb773 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:24.995152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfe393-89f9-49df-b29d-4b62c5cca07f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:24.996249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfe393-89f9-49df-b29d-4b62c5cca07f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:24.996772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfe393-89f9-49df-b29d-4b62c5cca07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:25.072273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c8cfe393-89f9-49df-b29d-4b62c5cca07f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:25.074800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c8cfe393-89f9-49df-b29d-4b62c5cca07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:25.248911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1117/2248] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:51:25.252639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc4289-bfec-4626-92c4-2895ffa5249b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:25.253044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc4289-bfec-4626-92c4-2895ffa5249b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:25.253443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc4289-bfec-4626-92c4-2895ffa5249b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:25.287908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-52bc4289-bfec-4626-92c4-2895ffa5249b None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:25.289804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-52bc4289-bfec-4626-92c4-2895ffa5249b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:25.322204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfe393-89f9-49df-b29d-4b62c5cca07f None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:25.329756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1132/2249] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:25.334460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfe393-89f9-49df-b29d-4b62c5cca07f None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:25.343309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc063f-f5fe-451f-8fca-d70e74d83764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:25.344874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc063f-f5fe-451f-8fca-d70e74d83764 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:25.345572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc063f-f5fe-451f-8fca-d70e74d83764 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:25.346351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc063f-f5fe-451f-8fca-d70e74d83764 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:25.347215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc063f-f5fe-451f-8fca-d70e74d83764 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:51:25.348006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc063f-f5fe-451f-8fca-d70e74d83764 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:25.353036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc063f-f5fe-451f-8fca-d70e74d83764 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:25.383545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc4289-bfec-4626-92c4-2895ffa5249b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:25.392087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1118/2250] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:51:25.400270 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc4289-bfec-4626-92c4-2895ffa5249b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:25.417615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89bbbc-6772-4d14-9cb7-c28596df4338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:25.418887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89bbbc-6772-4d14-9cb7-c28596df4338 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:25.419433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89bbbc-6772-4d14-9cb7-c28596df4338 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:25.420049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89bbbc-6772-4d14-9cb7-c28596df4338 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:25.421016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89bbbc-6772-4d14-9cb7-c28596df4338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:25.423662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89bbbc-6772-4d14-9cb7-c28596df4338 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:25.459255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1133/2251] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:51:25 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:51:25.463731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc063f-f5fe-451f-8fca-d70e74d83764 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:25.476326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc063f-f5fe-451f-8fca-d70e74d83764 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:25.578205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e85b62-3391-4146-ae28-f7c0de9d6620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:25.578937 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e85b62-3391-4146-ae28-f7c0de9d6620 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:25.579222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e85b62-3391-4146-ae28-f7c0de9d6620 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:25.579499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e85b62-3391-4146-ae28-f7c0de9d6620 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:25.580193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e85b62-3391-4146-ae28-f7c0de9d6620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:25.582331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e85b62-3391-4146-ae28-f7c0de9d6620 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:25.612664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1119/2252] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:51:25 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:51:25.614985 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89bbbc-6772-4d14-9cb7-c28596df4338 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:25.628118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89bbbc-6772-4d14-9cb7-c28596df4338 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:25.633405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9f04d-b34c-4e1c-96cb-fd42885571ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:25.634522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9f04d-b34c-4e1c-96cb-fd42885571ca admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:25.635015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9f04d-b34c-4e1c-96cb-fd42885571ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:25.635456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9f04d-b34c-4e1c-96cb-fd42885571ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:25.636651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9f04d-b34c-4e1c-96cb-fd42885571ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:25.639433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9f04d-b34c-4e1c-96cb-fd42885571ca admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:25.665141 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e85b62-3391-4146-ae28-f7c0de9d6620 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:25.665899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1134/2253] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:51:25.686162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e85b62-3391-4146-ae28-f7c0de9d6620 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:25.693194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a289da-0581-4bae-baf0-8bfd1b6c0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:25.693908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a289da-0581-4bae-baf0-8bfd1b6c0426 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:25.694430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a289da-0581-4bae-baf0-8bfd1b6c0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:25.694728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a289da-0581-4bae-baf0-8bfd1b6c0426 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:25.695463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a289da-0581-4bae-baf0-8bfd1b6c0426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:25.698611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a289da-0581-4bae-baf0-8bfd1b6c0426 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:25.722104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9f04d-b34c-4e1c-96cb-fd42885571ca admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:25.728807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1120/2254] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:51:25.736315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9f04d-b34c-4e1c-96cb-fd42885571ca admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:25.741799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46815855-69c9-4ca8-8441-9da3c9db08c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:25.742702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46815855-69c9-4ca8-8441-9da3c9db08c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:25.743000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46815855-69c9-4ca8-8441-9da3c9db08c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:25.743422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46815855-69c9-4ca8-8441-9da3c9db08c3 admin admin] PATH_INFO: `/v3/projects/5fd334ccd9b24de79d13c64ca71ca79f/users/14f70b324fb1419ca8733f9beba2021f/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:25.763185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46815855-69c9-4ca8-8441-9da3c9db08c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd334ccd9b24de79d13c64ca71ca79f, user_id=14f70b324fb1419ca8733f9beba2021f, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:25.767282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46815855-69c9-4ca8-8441-9da3c9db08c3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:25.785117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1135/2255] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:51:25 2022] PUT /identity/v3/projects/5fd334ccd9b24de79d13c64ca71ca79f/users/14f70b324fb1419ca8733f9beba2021f/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:25.797208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88810c2-d454-4ca9-bdcf-f1b82ee8ef41 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:25.797555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88810c2-d454-4ca9-bdcf-f1b82ee8ef41 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:25.797784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88810c2-d454-4ca9-bdcf-f1b82ee8ef41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:25.820858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:25.851979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a88810c2-d454-4ca9-bdcf-f1b82ee8ef41 None None] MFA Rules not processed for user `14f70b324fb1419ca8733f9beba2021f`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:25.853258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a88810c2-d454-4ca9-bdcf-f1b82ee8ef41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:26.040903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1121/2256] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:26.045870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46815855-69c9-4ca8-8441-9da3c9db08c3 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:26.101624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4812752-4ab4-41be-99a7-fa105f4a2ff9 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:26.102439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4812752-4ab4-41be-99a7-fa105f4a2ff9 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:26.102877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4812752-4ab4-41be-99a7-fa105f4a2ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:26.128562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:26.134518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a4812752-4ab4-41be-99a7-fa105f4a2ff9 None None] MFA Rules not processed for user `04e35f840ef045f0b44fab3e57a1a3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:26.138072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a4812752-4ab4-41be-99a7-fa105f4a2ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:26.204958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46815855-69c9-4ca8-8441-9da3c9db08c3 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:26.219520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718b451-5889-403a-91b4-99cfd8d66c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:26.220398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b451-5889-403a-91b4-99cfd8d66c72 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:26.220869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b451-5889-403a-91b4-99cfd8d66c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:26.221305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b451-5889-403a-91b4-99cfd8d66c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:26.222389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718b451-5889-403a-91b4-99cfd8d66c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:26.224532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718b451-5889-403a-91b4-99cfd8d66c72 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:26.365214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1136/2257] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:51:26 2022] POST /identity/v3/projects => generated 375 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:51:26.365981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718b451-5889-403a-91b4-99cfd8d66c72 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:26.422232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1122/2258] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:26.517402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718b451-5889-403a-91b4-99cfd8d66c72 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:26.553054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d897864-d88c-4141-a976-62153b223136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:26.554032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d897864-d88c-4141-a976-62153b223136 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:26.554529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d897864-d88c-4141-a976-62153b223136 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:26.555015 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d897864-d88c-4141-a976-62153b223136 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:26.556020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d897864-d88c-4141-a976-62153b223136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:26.561124 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d897864-d88c-4141-a976-62153b223136 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:26.637114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1137/2259] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:51:26 2022] POST /identity/v3/users => generated 294 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:51:26.640854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4812752-4ab4-41be-99a7-fa105f4a2ff9 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:26.657564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4812752-4ab4-41be-99a7-fa105f4a2ff9 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:26.669889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d1fd1-18c8-4e87-bff9-112bf0d94913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:26.674813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d1fd1-18c8-4e87-bff9-112bf0d94913 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:26.675341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d1fd1-18c8-4e87-bff9-112bf0d94913 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:26.675784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d1fd1-18c8-4e87-bff9-112bf0d94913 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:26.676766 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d1fd1-18c8-4e87-bff9-112bf0d94913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:26.678563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d1fd1-18c8-4e87-bff9-112bf0d94913 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:26.704442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1123/2260] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:51:26.708919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d897864-d88c-4141-a976-62153b223136 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:26.722604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d897864-d88c-4141-a976-62153b223136 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:26.732338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80698da5-b9d4-4ef8-9705-26323b01acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:26.733685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80698da5-b9d4-4ef8-9705-26323b01acfd admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:26.733976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80698da5-b9d4-4ef8-9705-26323b01acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:26.734463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80698da5-b9d4-4ef8-9705-26323b01acfd admin admin] PATH_INFO: `/v3/projects/726bde489a324850b922b979cac2ec96/users/bc9df0943b5a460d8503e8acbc57b865/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:26.754779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80698da5-b9d4-4ef8-9705-26323b01acfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=726bde489a324850b922b979cac2ec96, user_id=bc9df0943b5a460d8503e8acbc57b865, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:26.757508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80698da5-b9d4-4ef8-9705-26323b01acfd admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:26.789068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1138/2261] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:51:26 2022] PUT /identity/v3/projects/726bde489a324850b922b979cac2ec96/users/bc9df0943b5a460d8503e8acbc57b865/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:26.792882 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d1fd1-18c8-4e87-bff9-112bf0d94913 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:26.814811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d1fd1-18c8-4e87-bff9-112bf0d94913 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:26.946806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e416e7d-e2d2-4631-b9a0-167f8fb68cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:26.947494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e416e7d-e2d2-4631-b9a0-167f8fb68cac admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:26.947773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e416e7d-e2d2-4631-b9a0-167f8fb68cac admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:26.948051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e416e7d-e2d2-4631-b9a0-167f8fb68cac admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:26.948810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e416e7d-e2d2-4631-b9a0-167f8fb68cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:26.960919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e416e7d-e2d2-4631-b9a0-167f8fb68cac admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:26.993277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1124/2262] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:51:26.994589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80698da5-b9d4-4ef8-9705-26323b01acfd admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:27.009595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80698da5-b9d4-4ef8-9705-26323b01acfd admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:27.021109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc37ec-7a4d-4e06-9d42-20e3d0e9aa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:27.023525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc37ec-7a4d-4e06-9d42-20e3d0e9aa0a admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:27.024347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc37ec-7a4d-4e06-9d42-20e3d0e9aa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:27.024730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc37ec-7a4d-4e06-9d42-20e3d0e9aa0a admin admin] PATH_INFO: `/v3/projects/726bde489a324850b922b979cac2ec96/users/bc9df0943b5a460d8503e8acbc57b865/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:27.031994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc37ec-7a4d-4e06-9d42-20e3d0e9aa0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=726bde489a324850b922b979cac2ec96, user_id=bc9df0943b5a460d8503e8acbc57b865, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:27.034882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc37ec-7a4d-4e06-9d42-20e3d0e9aa0a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:27.076618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1139/2263] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:51:26 2022] PUT /identity/v3/projects/726bde489a324850b922b979cac2ec96/users/bc9df0943b5a460d8503e8acbc57b865/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:27.084540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8744f-7b9e-4403-8336-9491ea42901b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:27.084856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8744f-7b9e-4403-8336-9491ea42901b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:27.085135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8744f-7b9e-4403-8336-9491ea42901b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:27.147773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-7cf8744f-7b9e-4403-8336-9491ea42901b None None] MFA Rules not processed for user `bc9df0943b5a460d8503e8acbc57b865`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:27.150931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7cf8744f-7b9e-4403-8336-9491ea42901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:27.177146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:27.350418 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1125/2264] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:27.369685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc37ec-7a4d-4e06-9d42-20e3d0e9aa0a admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:27.371730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ebbc37ec-7a4d-4e06-9d42-20e3d0e9aa0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:27.567853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc37ec-7a4d-4e06-9d42-20e3d0e9aa0a admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:27.574778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db2b4b-dcb9-4b5a-982a-25902bdc7dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:27.576254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2b4b-dcb9-4b5a-982a-25902bdc7dcc service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:27.577403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2b4b-dcb9-4b5a-982a-25902bdc7dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:27.577714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2b4b-dcb9-4b5a-982a-25902bdc7dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:27.605983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:27.696469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db2b4b-dcb9-4b5a-982a-25902bdc7dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:27.698665 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db2b4b-dcb9-4b5a-982a-25902bdc7dcc service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:27.711451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1140/2265] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:32.815222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8744f-7b9e-4403-8336-9491ea42901b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:32.942742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8744f-7b9e-4403-8336-9491ea42901b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:32.948112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:32.950789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:32.951418 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:32.951418 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] PATH_INFO: `/v3/users/14f70b324fb1419ca8733f9beba2021f` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:32.952677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14f70b324fb1419ca8733f9beba2021f)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:32.954644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:33.033547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14f70b324fb1419ca8733f9beba2021f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:33.047665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14f70b324fb1419ca8733f9beba2021f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:33.057941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14f70b324fb1419ca8733f9beba2021f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:33.059948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14f70b324fb1419ca8733f9beba2021f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:33.069748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14f70b324fb1419ca8733f9beba2021f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:33.078157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1126/2266] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:51:32 2022] DELETE /identity/v3/users/14f70b324fb1419ca8733f9beba2021f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:33.083147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c2368-8d78-49a9-814f-f87a0a352c0f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:33.083689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c2368-8d78-49a9-814f-f87a0a352c0f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:33.083689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c2368-8d78-49a9-814f-f87a0a352c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:33.104560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f93c2368-8d78-49a9-814f-f87a0a352c0f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:33.107106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f93c2368-8d78-49a9-814f-f87a0a352c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:33.224706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1141/2267] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:51:33.239301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:33.241139 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:33.314571 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:33.348952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6662ff3-6425-42ed-83a8-0b4ca2b4b4f2 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:33.353612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775874f8-9968-449c-b551-6b4698710ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:33.354397 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775874f8-9968-449c-b551-6b4698710ffd service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:33.354782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775874f8-9968-449c-b551-6b4698710ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:33.355296 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775874f8-9968-449c-b551-6b4698710ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:33.372501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775874f8-9968-449c-b551-6b4698710ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:33.373828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775874f8-9968-449c-b551-6b4698710ffd service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:33.385301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1127/2268] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:34.189869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c2368-8d78-49a9-814f-f87a0a352c0f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:34.192248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f93c2368-8d78-49a9-814f-f87a0a352c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:34.228109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c2368-8d78-49a9-814f-f87a0a352c0f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:34.238986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:34.240121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:34.240801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:34.241309 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] PATH_INFO: `/v3/projects/5fd334ccd9b24de79d13c64ca71ca79f` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:34.242730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd334ccd9b24de79d13c64ca71ca79f)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:34.244878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:34.345769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd334ccd9b24de79d13c64ca71ca79f', 'request_id': 'req-645207f4-6b51-4576-9766-a3174ad7ff07'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.365454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd334ccd9b24de79d13c64ca71ca79f', 'request_id': 'req-645207f4-6b51-4576-9766-a3174ad7ff07'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.374521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd334ccd9b24de79d13c64ca71ca79f', 'request_id': 'req-645207f4-6b51-4576-9766-a3174ad7ff07'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.391126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] The token cache is being invalidate because project 5fd334ccd9b24de79d13c64ca71ca79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:51:34.391554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.392308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.397435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1142/2269] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:51:34 2022] DELETE /identity/v3/projects/5fd334ccd9b24de79d13c64ca71ca79f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:34.398658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775874f8-9968-449c-b551-6b4698710ffd service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:34.411289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775874f8-9968-449c-b551-6b4698710ffd service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:34.487217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:34.488606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:34.488913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:34.489196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] PATH_INFO: `/v3/users/bc9df0943b5a460d8503e8acbc57b865` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:34.490535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc9df0943b5a460d8503e8acbc57b865)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:34.492781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:34.553858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc9df0943b5a460d8503e8acbc57b865'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.572041 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc9df0943b5a460d8503e8acbc57b865'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.582074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc9df0943b5a460d8503e8acbc57b865'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.582756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc9df0943b5a460d8503e8acbc57b865'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.590389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ec922793-99b2-4988-a7df-f055e2043ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc9df0943b5a460d8503e8acbc57b865'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:34.597832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1128/2270] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:51:34 2022] DELETE /identity/v3/users/bc9df0943b5a460d8503e8acbc57b865 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:35.096326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:35.097993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:35.127717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645207f4-6b51-4576-9766-a3174ad7ff07 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:35.193033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:35.224667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:35.226281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.228697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.229152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] PATH_INFO: `/v3/projects/726bde489a324850b922b979cac2ec96` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:35.230809 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=726bde489a324850b922b979cac2ec96)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:35.234755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:35.306768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726bde489a324850b922b979cac2ec96', 'request_id': 'req-50e0c645-899f-4634-bda8-6705e0bc9526'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:35.318293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726bde489a324850b922b979cac2ec96', 'request_id': 'req-50e0c645-899f-4634-bda8-6705e0bc9526'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:35.329490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726bde489a324850b922b979cac2ec96', 'request_id': 'req-50e0c645-899f-4634-bda8-6705e0bc9526'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:35.366169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] The token cache is being invalidate because project 726bde489a324850b922b979cac2ec96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:51:35.366502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:35.368623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-50e0c645-899f-4634-bda8-6705e0bc9526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:51:35.376628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1143/2271] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:51:35 2022] DELETE /identity/v3/projects/726bde489a324850b922b979cac2ec96 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:35.376777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490eb24-f264-4c8d-a4ba-c514eb41b44a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.377173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490eb24-f264-4c8d-a4ba-c514eb41b44a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.378495 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490eb24-f264-4c8d-a4ba-c514eb41b44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:35.410612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-4490eb24-f264-4c8d-a4ba-c514eb41b44a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:35.412957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4490eb24-f264-4c8d-a4ba-c514eb41b44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:35.509476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:35.528957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1129/2272] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:51:35.538169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04d5c5-f391-4b95-8cb4-3c70efffdbe1 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.538703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04d5c5-f391-4b95-8cb4-3c70efffdbe1 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.538989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04d5c5-f391-4b95-8cb4-3c70efffdbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:35.563172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-cf04d5c5-f391-4b95-8cb4-3c70efffdbe1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:35.564979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-cf04d5c5-f391-4b95-8cb4-3c70efffdbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:35.575394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1144/2273] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:35.579897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490eb24-f264-4c8d-a4ba-c514eb41b44a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:35.594783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490eb24-f264-4c8d-a4ba-c514eb41b44a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:35.601249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0587db0-76b1-44e3-9562-b73517d0fe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:35.602305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0587db0-76b1-44e3-9562-b73517d0fe83 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.602783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0587db0-76b1-44e3-9562-b73517d0fe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.603264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0587db0-76b1-44e3-9562-b73517d0fe83 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:35.604531 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0587db0-76b1-44e3-9562-b73517d0fe83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:51:35.605090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0587db0-76b1-44e3-9562-b73517d0fe83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:35.607484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0587db0-76b1-44e3-9562-b73517d0fe83 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:35.621956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1130/2274] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:51:35.626548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04d5c5-f391-4b95-8cb4-3c70efffdbe1 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:35.638242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04d5c5-f391-4b95-8cb4-3c70efffdbe1 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:35.643975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859123ac-5f82-4ac5-8878-e11a0a3f3862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:35.644680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859123ac-5f82-4ac5-8878-e11a0a3f3862 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.644972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859123ac-5f82-4ac5-8878-e11a0a3f3862 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.645255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859123ac-5f82-4ac5-8878-e11a0a3f3862 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:35.645956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859123ac-5f82-4ac5-8878-e11a0a3f3862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:35.648382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859123ac-5f82-4ac5-8878-e11a0a3f3862 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:35.691482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1145/2275] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:51:35 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:51:35.692750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0587db0-76b1-44e3-9562-b73517d0fe83 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:35.706684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0587db0-76b1-44e3-9562-b73517d0fe83 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:35.782271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57006015-7cae-4903-bb7f-30bf88bfd190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:35.783032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57006015-7cae-4903-bb7f-30bf88bfd190 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.783330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57006015-7cae-4903-bb7f-30bf88bfd190 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.783610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57006015-7cae-4903-bb7f-30bf88bfd190 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:35.784338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57006015-7cae-4903-bb7f-30bf88bfd190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:35.786291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57006015-7cae-4903-bb7f-30bf88bfd190 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:35.815785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1131/2276] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:51:35 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:51:35.819467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859123ac-5f82-4ac5-8878-e11a0a3f3862 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:35.831709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859123ac-5f82-4ac5-8878-e11a0a3f3862 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:35.836728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8a088-55af-4fb3-9f73-33fbdb8883ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:35.837390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8a088-55af-4fb3-9f73-33fbdb8883ff admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.837674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8a088-55af-4fb3-9f73-33fbdb8883ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.837949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8a088-55af-4fb3-9f73-33fbdb8883ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:35.838676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8a088-55af-4fb3-9f73-33fbdb8883ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:35.840276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8a088-55af-4fb3-9f73-33fbdb8883ff admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:35.856644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1146/2277] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:51:35.858047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57006015-7cae-4903-bb7f-30bf88bfd190 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:35.868890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57006015-7cae-4903-bb7f-30bf88bfd190 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:35.874242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a38864-549b-46bc-92f2-9369eee85ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:35.874916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a38864-549b-46bc-92f2-9369eee85ffe admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.875203 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a38864-549b-46bc-92f2-9369eee85ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.875652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a38864-549b-46bc-92f2-9369eee85ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:35.876226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a38864-549b-46bc-92f2-9369eee85ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:35.877917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a38864-549b-46bc-92f2-9369eee85ffe admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:35.892978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1132/2278] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:51:35.895318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8a088-55af-4fb3-9f73-33fbdb8883ff admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:35.922795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8a088-55af-4fb3-9f73-33fbdb8883ff admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:35.928353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb644894-b515-4fd9-9f98-f505349ebfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:35.929728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb644894-b515-4fd9-9f98-f505349ebfbc admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.930096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb644894-b515-4fd9-9f98-f505349ebfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.930450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb644894-b515-4fd9-9f98-f505349ebfbc admin admin] PATH_INFO: `/v3/projects/9f579ab134df4b9f9ea9f6e2c035a7fb/users/09c95554a3ea4476b2ce834b33de2bbf/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:35.952571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb644894-b515-4fd9-9f98-f505349ebfbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f579ab134df4b9f9ea9f6e2c035a7fb, user_id=09c95554a3ea4476b2ce834b33de2bbf, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:35.956797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb644894-b515-4fd9-9f98-f505349ebfbc admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:35.979469 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1147/2279] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:51:35 2022] PUT /identity/v3/projects/9f579ab134df4b9f9ea9f6e2c035a7fb/users/09c95554a3ea4476b2ce834b33de2bbf/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:51:35.985645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e660d-a426-49cb-93d7-e59c9dfd3bca None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:35.985963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e660d-a426-49cb-93d7-e59c9dfd3bca None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:35.986268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e660d-a426-49cb-93d7-e59c9dfd3bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:36.011881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-6c5e660d-a426-49cb-93d7-e59c9dfd3bca None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:36.013602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6c5e660d-a426-49cb-93d7-e59c9dfd3bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:36.133049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1133/2280] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:36.156110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb644894-b515-4fd9-9f98-f505349ebfbc admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:36.158544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-eb644894-b515-4fd9-9f98-f505349ebfbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:36.240856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb644894-b515-4fd9-9f98-f505349ebfbc admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:36.245494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc796e8f-05e0-4b09-9165-d4e08415a244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:36.246328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc796e8f-05e0-4b09-9165-d4e08415a244 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:36.247419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc796e8f-05e0-4b09-9165-d4e08415a244 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:36.247874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc796e8f-05e0-4b09-9165-d4e08415a244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:36.261526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc796e8f-05e0-4b09-9165-d4e08415a244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:36.265333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc796e8f-05e0-4b09-9165-d4e08415a244 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:36.279929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1148/2281] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:51:37.068358 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592d8e2-feeb-4ab9-b9b3-9d1b618a38f6 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:37.068933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592d8e2-feeb-4ab9-b9b3-9d1b618a38f6 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:37.069029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592d8e2-feeb-4ab9-b9b3-9d1b618a38f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:37.091895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-b592d8e2-feeb-4ab9-b9b3-9d1b618a38f6 None None] MFA Rules not processed for user `6e416196a4614ccc90c8f2411e3f0258`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:37.093543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b592d8e2-feeb-4ab9-b9b3-9d1b618a38f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:37.176500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1134/2282] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:37.506846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a1686-2343-48bd-a8e6-04bb4e51dbeb None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:37.507404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a1686-2343-48bd-a8e6-04bb4e51dbeb None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:37.507404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a1686-2343-48bd-a8e6-04bb4e51dbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:37.527543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:37.565844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-711a1686-2343-48bd-a8e6-04bb4e51dbeb None None] MFA Rules not processed for user `09c95554a3ea4476b2ce834b33de2bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:37.567785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-711a1686-2343-48bd-a8e6-04bb4e51dbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:37.669973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1149/2283] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:37.680285 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592d8e2-feeb-4ab9-b9b3-9d1b618a38f6 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:37.682442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b592d8e2-feeb-4ab9-b9b3-9d1b618a38f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:37.707725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:37.801304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592d8e2-feeb-4ab9-b9b3-9d1b618a38f6 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:37.807221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45eaeb9-db6a-4fc9-8979-6b878a459a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:37.808364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eaeb9-db6a-4fc9-8979-6b878a459a71 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:37.808364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eaeb9-db6a-4fc9-8979-6b878a459a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:37.808712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45eaeb9-db6a-4fc9-8979-6b878a459a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:37.824299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eaeb9-db6a-4fc9-8979-6b878a459a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:37.828661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45eaeb9-db6a-4fc9-8979-6b878a459a71 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:37.845211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1135/2284] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:39.131461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a1686-2343-48bd-a8e6-04bb4e51dbeb None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:39.133046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-711a1686-2343-48bd-a8e6-04bb4e51dbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:39.270589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a1686-2343-48bd-a8e6-04bb4e51dbeb None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:39.276968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e18a30-1d86-44a7-8f83-3a3ea19eb465 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:39.277693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e18a30-1d86-44a7-8f83-3a3ea19eb465 service placement] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:39.278529 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e18a30-1d86-44a7-8f83-3a3ea19eb465 service placement] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:39.279052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e18a30-1d86-44a7-8f83-3a3ea19eb465 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:39.282279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d2e18a30-1d86-44a7-8f83-3a3ea19eb465 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:39.298783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e18a30-1d86-44a7-8f83-3a3ea19eb465 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:39.300443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e18a30-1d86-44a7-8f83-3a3ea19eb465 service placement] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:39.311977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1150/2285] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:39.573755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3cab3-d06d-4f3b-9866-274b4845f871 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:39.574316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3cab3-d06d-4f3b-9866-274b4845f871 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:39.574316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3cab3-d06d-4f3b-9866-274b4845f871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:39.617857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-38b3cab3-d06d-4f3b-9866-274b4845f871 None None] MFA Rules not processed for user `09c95554a3ea4476b2ce834b33de2bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:39.619360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-38b3cab3-d06d-4f3b-9866-274b4845f871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:39.644477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1136/2286] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:51:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:44.327779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e18a30-1d86-44a7-8f83-3a3ea19eb465 service placement] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:44.338933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e18a30-1d86-44a7-8f83-3a3ea19eb465 service placement] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:44.343993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71b70c-efbf-4026-a5bf-f6fec6ce4cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:44.344805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71b70c-efbf-4026-a5bf-f6fec6ce4cfd service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:44.345250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71b70c-efbf-4026-a5bf-f6fec6ce4cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:44.345650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71b70c-efbf-4026-a5bf-f6fec6ce4cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:44.347815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0a71b70c-efbf-4026-a5bf-f6fec6ce4cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:44.417418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71b70c-efbf-4026-a5bf-f6fec6ce4cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:44.419775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71b70c-efbf-4026-a5bf-f6fec6ce4cfd service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:44.429358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1151/2287] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:47.074797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68757f7a-a723-4af6-9949-18e84d9e1589 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:47.076133 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68757f7a-a723-4af6-9949-18e84d9e1589 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:47.076440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68757f7a-a723-4af6-9949-18e84d9e1589 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:47.077868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1137/2288] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:51:47.085188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a816d3-a37c-4459-8eb4-fa4399747931 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:47.085706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a816d3-a37c-4459-8eb4-fa4399747931 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:47.085821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a816d3-a37c-4459-8eb4-fa4399747931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:47.093366 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:51:47.151573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-30a816d3-a37c-4459-8eb4-fa4399747931 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:47.153643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-30a816d3-a37c-4459-8eb4-fa4399747931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:47.311412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1152/2289] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:51:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:47.323290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68757f7a-a723-4af6-9949-18e84d9e1589 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:47.326506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-68757f7a-a723-4af6-9949-18e84d9e1589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:47.401497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68757f7a-a723-4af6-9949-18e84d9e1589 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:47.407165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393552be-5862-4085-b7be-675bb8afbd5b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71e5ef9dc32f494791ba4addb5151232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:47.407849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393552be-5862-4085-b7be-675bb8afbd5b service glance] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:47.408132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393552be-5862-4085-b7be-675bb8afbd5b service glance] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:47.408415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393552be-5862-4085-b7be-675bb8afbd5b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:47.421417 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393552be-5862-4085-b7be-675bb8afbd5b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:47.422885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393552be-5862-4085-b7be-675bb8afbd5b service glance] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:47.436469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1138/2290] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:47.547347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d62c3-1ae1-485b-99f5-259d9074ce3b None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:47.547698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d62c3-1ae1-485b-99f5-259d9074ce3b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:47.548047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d62c3-1ae1-485b-99f5-259d9074ce3b None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:47.550394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1153/2291] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:51:47.556493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f6249-ea34-417b-b630-736c13b42f53 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:47.557003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f6249-ea34-417b-b630-736c13b42f53 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:47.557456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f6249-ea34-417b-b630-736c13b42f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:47.613318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-442f6249-ea34-417b-b630-736c13b42f53 None None] MFA Rules not processed for user `e38a311f2b614a3598f6b51b3a235178`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:47.617097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-442f6249-ea34-417b-b630-736c13b42f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:47.638063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1139/2292] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:51:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:47.676085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d62c3-1ae1-485b-99f5-259d9074ce3b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:47.696740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d62c3-1ae1-485b-99f5-259d9074ce3b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:47.708669 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce5b12-edba-4b9c-9350-b818bb998efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:47.709304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce5b12-edba-4b9c-9350-b818bb998efe service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:47.709591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce5b12-edba-4b9c-9350-b818bb998efe service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:47.709864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce5b12-edba-4b9c-9350-b818bb998efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:47.722914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce5b12-edba-4b9c-9350-b818bb998efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:47.724562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce5b12-edba-4b9c-9350-b818bb998efe service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:47.749004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1154/2293] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:51:47.881159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98afa82-2f86-4dcc-9ba8-7b987af7d5d4 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:47.881813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98afa82-2f86-4dcc-9ba8-7b987af7d5d4 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:47.882332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98afa82-2f86-4dcc-9ba8-7b987af7d5d4 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:47.886446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1140/2294] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:51:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:51:47.905564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74972e83-d3b5-4b96-b5f6-0329f60b24a5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:47.905926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74972e83-d3b5-4b96-b5f6-0329f60b24a5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:47.906142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74972e83-d3b5-4b96-b5f6-0329f60b24a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:47.964092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-74972e83-d3b5-4b96-b5f6-0329f60b24a5 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:47.967388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-74972e83-d3b5-4b96-b5f6-0329f60b24a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:47.986091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1155/2295] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:51:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:48.033926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98afa82-2f86-4dcc-9ba8-7b987af7d5d4 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:48.047255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98afa82-2f86-4dcc-9ba8-7b987af7d5d4 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:48.051699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d527e5-1ec1-4bea-a210-7c1d03c1b8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:48.052414 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d527e5-1ec1-4bea-a210-7c1d03c1b8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:48.052737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d527e5-1ec1-4bea-a210-7c1d03c1b8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:48.053061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d527e5-1ec1-4bea-a210-7c1d03c1b8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:48.073021 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d527e5-1ec1-4bea-a210-7c1d03c1b8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:48.075252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d527e5-1ec1-4bea-a210-7c1d03c1b8e4 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:48.090258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1141/2296] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:51:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:51:56.287475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d71a0-d541-4b84-9e61-c3c5d888f95e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:56.287475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d71a0-d541-4b84-9e61-c3c5d888f95e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:56.287475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d71a0-d541-4b84-9e61-c3c5d888f95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:56.304418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-272d71a0-d541-4b84-9e61-c3c5d888f95e None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:51:56.305890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-272d71a0-d541-4b84-9e61-c3c5d888f95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:56.363945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1156/2297] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:51:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:51:56.387280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d527e5-1ec1-4bea-a210-7c1d03c1b8e4 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:51:56.388785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f9d527e5-1ec1-4bea-a210-7c1d03c1b8e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:51:56.467692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d527e5-1ec1-4bea-a210-7c1d03c1b8e4 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:51:56.472883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4a20d-a0b2-4c14-bb7b-decefa89c8cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:51:56.473533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4a20d-a0b2-4c14-bb7b-decefa89c8cc service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:51:56.473820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4a20d-a0b2-4c14-bb7b-decefa89c8cc service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:51:56.474095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4a20d-a0b2-4c14-bb7b-decefa89c8cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:51:56.483607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4a20d-a0b2-4c14-bb7b-decefa89c8cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:51:56.485236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4a20d-a0b2-4c14-bb7b-decefa89c8cc service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:51:56.493675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1142/2298] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:51:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:52:06.357240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=81 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:52:06.490157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=93 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:52:18.135268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd510769-2c7c-4df5-88c1-2a833d076e5e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:52:18.135785 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd510769-2c7c-4df5-88c1-2a833d076e5e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:52:18.135896 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd510769-2c7c-4df5-88c1-2a833d076e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:52:18.154094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-cd510769-2c7c-4df5-88c1-2a833d076e5e None None] MFA Rules not processed for user `09c95554a3ea4476b2ce834b33de2bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:52:18.155327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-cd510769-2c7c-4df5-88c1-2a833d076e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:52:18.162112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1157/2299] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:52:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:52:28.148867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:52:29.610552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac89bc-47d3-4a18-92ca-29fcdc39fcf5 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:52:29.611114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac89bc-47d3-4a18-92ca-29fcdc39fcf5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:52:29.611114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac89bc-47d3-4a18-92ca-29fcdc39fcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:52:29.662588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-b0ac89bc-47d3-4a18-92ca-29fcdc39fcf5 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:52:29.663595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b0ac89bc-47d3-4a18-92ca-29fcdc39fcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:52:29.682690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1143/2300] 10.222.0.47 () {58 vars in 967 bytes} [Fri Jun 24 23:52:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:52:29.695459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd510769-2c7c-4df5-88c1-2a833d076e5e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:52:29.717204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd510769-2c7c-4df5-88c1-2a833d076e5e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:52:29.725922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef274f3e-8763-4a7d-8708-98ac2db43ad7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:52:29.726719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef274f3e-8763-4a7d-8708-98ac2db43ad7 service swift] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:52:29.727071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef274f3e-8763-4a7d-8708-98ac2db43ad7 service swift] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:52:29.727424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef274f3e-8763-4a7d-8708-98ac2db43ad7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:52:29.738541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef274f3e-8763-4a7d-8708-98ac2db43ad7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:52:29.740337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef274f3e-8763-4a7d-8708-98ac2db43ad7 service swift] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:52:29.753919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1158/2301] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:52:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:52:39.679428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=18 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:52:39.746827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=10 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:52:46.416528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b68a4e-d24c-480c-a67d-c2322c6a2e74 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:52:46.417063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b68a4e-d24c-480c-a67d-c2322c6a2e74 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:52:46.417416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b68a4e-d24c-480c-a67d-c2322c6a2e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:52:46.435552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c9b68a4e-d24c-480c-a67d-c2322c6a2e74 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:52:46.437315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c9b68a4e-d24c-480c-a67d-c2322c6a2e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:52:46.444651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1144/2302] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:52:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:52:46.457693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef274f3e-8763-4a7d-8708-98ac2db43ad7 service swift] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:52:46.469498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef274f3e-8763-4a7d-8708-98ac2db43ad7 service swift] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:52:46.474999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6470ea-0e2e-40a8-9b6f-97a5204ee27d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:52:46.475742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6470ea-0e2e-40a8-9b6f-97a5204ee27d service swift] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:52:46.476088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6470ea-0e2e-40a8-9b6f-97a5204ee27d service swift] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:52:46.476428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6470ea-0e2e-40a8-9b6f-97a5204ee27d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:52:46.485300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6470ea-0e2e-40a8-9b6f-97a5204ee27d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:52:46.487198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6470ea-0e2e-40a8-9b6f-97a5204ee27d service swift] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:52:46.495332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1159/2303] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:52:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:52:56.430240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:52:56.491974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=6 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:53:03.872038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcfba2-8f06-4c31-989e-bedf0bb80f08 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:03.872503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcfba2-8f06-4c31-989e-bedf0bb80f08 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:03.872613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbcfba2-8f06-4c31-989e-bedf0bb80f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:03.895412 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8fbcfba2-8f06-4c31-989e-bedf0bb80f08 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:53:03.896652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8fbcfba2-8f06-4c31-989e-bedf0bb80f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:03.906440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1145/2304] 10.222.0.47 () {58 vars in 967 bytes} [Fri Jun 24 23:53:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:53:03.922962 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6470ea-0e2e-40a8-9b6f-97a5204ee27d service swift] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:03.942165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6470ea-0e2e-40a8-9b6f-97a5204ee27d service swift] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:03.948046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c837772-14ed-4d0a-a2fd-6d8c5896784d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:03.956851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837772-14ed-4d0a-a2fd-6d8c5896784d service swift] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:03.956851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837772-14ed-4d0a-a2fd-6d8c5896784d service swift] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:03.956851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c837772-14ed-4d0a-a2fd-6d8c5896784d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:03.976014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c837772-14ed-4d0a-a2fd-6d8c5896784d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:03.976999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c837772-14ed-4d0a-a2fd-6d8c5896784d service swift] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:04.000128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1160/2305] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:53:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 23:53:04.428818 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcfba2-8f06-4c31-989e-bedf0bb80f08 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:04.430662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8fbcfba2-8f06-4c31-989e-bedf0bb80f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:04.448750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbcfba2-8f06-4c31-989e-bedf0bb80f08 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:04.450420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:04.451328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:04.451603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:04.452125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] PATH_INFO: `/v3/users/6e416196a4614ccc90c8f2411e3f0258` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:04.453565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e416196a4614ccc90c8f2411e3f0258)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:04.455500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:04.504877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e416196a4614ccc90c8f2411e3f0258'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.519228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e416196a4614ccc90c8f2411e3f0258'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.527274 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e416196a4614ccc90c8f2411e3f0258'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.527972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e416196a4614ccc90c8f2411e3f0258'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.536741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e416196a4614ccc90c8f2411e3f0258'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.543094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1146/2306] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:53:04 2022] DELETE /identity/v3/users/6e416196a4614ccc90c8f2411e3f0258 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:53:04.747085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c837772-14ed-4d0a-a2fd-6d8c5896784d service swift] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:04.759611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c837772-14ed-4d0a-a2fd-6d8c5896784d service swift] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:04.831417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:04.832281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:04.832562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:04.832867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] PATH_INFO: `/v3/projects/b45f691b51284dbbb7f1fd8d482af7e4` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:04.834052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b45f691b51284dbbb7f1fd8d482af7e4)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:04.835961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:04.886336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b45f691b51284dbbb7f1fd8d482af7e4', 'request_id': 'req-ace77765-cd27-47a2-823a-327dc68f210e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.895424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b45f691b51284dbbb7f1fd8d482af7e4', 'request_id': 'req-ace77765-cd27-47a2-823a-327dc68f210e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.903106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b45f691b51284dbbb7f1fd8d482af7e4', 'request_id': 'req-ace77765-cd27-47a2-823a-327dc68f210e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.917914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] The token cache is being invalidate because project b45f691b51284dbbb7f1fd8d482af7e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:53:04.918228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.918874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:04.920427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1161/2307] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:53:04 2022] DELETE /identity/v3/projects/b45f691b51284dbbb7f1fd8d482af7e4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:53:06.081911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:06.083455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:06.105925 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf8cd5-0363-4255-b1d6-7904dec54994 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:06.173517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:06.174573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:06.174950 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:06.175498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] PATH_INFO: `/v3/users/09c95554a3ea4476b2ce834b33de2bbf` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:06.176970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c95554a3ea4476b2ce834b33de2bbf)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:06.178970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:06.238299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c95554a3ea4476b2ce834b33de2bbf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.251747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c95554a3ea4476b2ce834b33de2bbf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.258430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:53:06.261079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c95554a3ea4476b2ce834b33de2bbf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.261785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c95554a3ea4476b2ce834b33de2bbf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.275952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-4575fa36-25cb-498e-bf79-e6c97e178509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c95554a3ea4476b2ce834b33de2bbf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.292674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1147/2308] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:53:06 2022] DELETE /identity/v3/users/09c95554a3ea4476b2ce834b33de2bbf => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:53:06.515725 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:06.518000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:06.542657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace77765-cd27-47a2-823a-327dc68f210e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:06.606571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:06.607569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:06.607850 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:06.608744 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] PATH_INFO: `/v3/projects/9f579ab134df4b9f9ea9f6e2c035a7fb` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:06.610139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f579ab134df4b9f9ea9f6e2c035a7fb)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:06.612390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:06.638669 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:53:06.675421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f579ab134df4b9f9ea9f6e2c035a7fb', 'request_id': 'req-d3559187-43e5-4bab-abbb-ee3003a4d4e4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.685098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f579ab134df4b9f9ea9f6e2c035a7fb', 'request_id': 'req-d3559187-43e5-4bab-abbb-ee3003a4d4e4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.693624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f579ab134df4b9f9ea9f6e2c035a7fb', 'request_id': 'req-d3559187-43e5-4bab-abbb-ee3003a4d4e4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.709247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] The token cache is being invalidate because project 9f579ab134df4b9f9ea9f6e2c035a7fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:53:06.709727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.710477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d3559187-43e5-4bab-abbb-ee3003a4d4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:06.712178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1162/2309] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:53:06 2022] DELETE /identity/v3/projects/9f579ab134df4b9f9ea9f6e2c035a7fb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:53:06.717544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975d23f-a641-42a8-899c-0262b7c4569c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:06.717909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975d23f-a641-42a8-899c-0262b7c4569c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:06.718193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975d23f-a641-42a8-899c-0262b7c4569c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:06.737794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-2975d23f-a641-42a8-899c-0262b7c4569c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:53:06.739016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2975d23f-a641-42a8-899c-0262b7c4569c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:06.801859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1148/2310] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:53:06.810824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114c44-3949-4105-aa02-45617da7d9ba None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:06.811643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114c44-3949-4105-aa02-45617da7d9ba None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:06.811930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c114c44-3949-4105-aa02-45617da7d9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:06.830563 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9c114c44-3949-4105-aa02-45617da7d9ba None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:53:06.831762 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9c114c44-3949-4105-aa02-45617da7d9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:06.841389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1163/2311] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:53:06.847543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975d23f-a641-42a8-899c-0262b7c4569c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:06.857549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975d23f-a641-42a8-899c-0262b7c4569c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:06.862455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566386b3-6317-4ea8-ade3-e465e708483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:06.863108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566386b3-6317-4ea8-ade3-e465e708483d admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:06.863389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566386b3-6317-4ea8-ade3-e465e708483d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:06.863664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566386b3-6317-4ea8-ade3-e465e708483d admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:06.864389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566386b3-6317-4ea8-ade3-e465e708483d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:53:06.864731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566386b3-6317-4ea8-ade3-e465e708483d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:06.866316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566386b3-6317-4ea8-ade3-e465e708483d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:06.882283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1149/2312] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:53:06.883945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c114c44-3949-4105-aa02-45617da7d9ba None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:06.893771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c114c44-3949-4105-aa02-45617da7d9ba None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:06.899466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708c689-17b5-4430-8522-b314232c28f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:06.900198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708c689-17b5-4430-8522-b314232c28f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:06.900483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708c689-17b5-4430-8522-b314232c28f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:06.900799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708c689-17b5-4430-8522-b314232c28f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:06.901756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708c689-17b5-4430-8522-b314232c28f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:06.903674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708c689-17b5-4430-8522-b314232c28f1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:06.935839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1164/2313] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:53:06 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:53:06.937483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566386b3-6317-4ea8-ade3-e465e708483d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:06.947964 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566386b3-6317-4ea8-ade3-e465e708483d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:07.013428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a4de4-9267-4e82-9924-c2d2193efa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:07.014103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a4de4-9267-4e82-9924-c2d2193efa36 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:07.014958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a4de4-9267-4e82-9924-c2d2193efa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:07.015445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a4de4-9267-4e82-9924-c2d2193efa36 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:07.016142 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a4de4-9267-4e82-9924-c2d2193efa36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:07.018114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a4de4-9267-4e82-9924-c2d2193efa36 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:07.041777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1150/2314] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:53:06 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:53:07.043029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708c689-17b5-4430-8522-b314232c28f1 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:07.054145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708c689-17b5-4430-8522-b314232c28f1 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:07.059167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50072283-f8e7-4b15-b3e8-95c4162a2b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:07.059999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50072283-f8e7-4b15-b3e8-95c4162a2b19 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:07.060350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50072283-f8e7-4b15-b3e8-95c4162a2b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:07.060742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50072283-f8e7-4b15-b3e8-95c4162a2b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:07.061691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50072283-f8e7-4b15-b3e8-95c4162a2b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:07.063245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50072283-f8e7-4b15-b3e8-95c4162a2b19 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:07.074512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1165/2315] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:53:07.079504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a4de4-9267-4e82-9924-c2d2193efa36 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:07.090626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a4de4-9267-4e82-9924-c2d2193efa36 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:07.103414 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8b894-5f0f-4ce8-8e5e-7d43733ea6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:07.104228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8b894-5f0f-4ce8-8e5e-7d43733ea6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:07.104607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8b894-5f0f-4ce8-8e5e-7d43733ea6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:07.104962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8b894-5f0f-4ce8-8e5e-7d43733ea6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:07.105781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8b894-5f0f-4ce8-8e5e-7d43733ea6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:07.109436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8b894-5f0f-4ce8-8e5e-7d43733ea6a2 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:07.122185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1151/2316] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:53:07.129787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50072283-f8e7-4b15-b3e8-95c4162a2b19 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:07.141507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50072283-f8e7-4b15-b3e8-95c4162a2b19 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:07.158688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758324a-0cc9-418f-ba94-ba62247388e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:07.159645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758324a-0cc9-418f-ba94-ba62247388e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:07.159930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758324a-0cc9-418f-ba94-ba62247388e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:07.160216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758324a-0cc9-418f-ba94-ba62247388e0 admin admin] PATH_INFO: `/v3/projects/38de3652d4e548faa55a8b0a860ab2e5/users/4759f79e6a2a443db1bfe2c8764a850c/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:07.177429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758324a-0cc9-418f-ba94-ba62247388e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38de3652d4e548faa55a8b0a860ab2e5, user_id=4759f79e6a2a443db1bfe2c8764a850c, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:07.180551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758324a-0cc9-418f-ba94-ba62247388e0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:07.199914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1166/2317] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:53:07 2022] PUT /identity/v3/projects/38de3652d4e548faa55a8b0a860ab2e5/users/4759f79e6a2a443db1bfe2c8764a850c/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:53:07.211475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea2b3-81f7-4466-a0f3-34e1bca5076d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:07.211795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea2b3-81f7-4466-a0f3-34e1bca5076d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:07.212108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea2b3-81f7-4466-a0f3-34e1bca5076d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:07.248568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-e17ea2b3-81f7-4466-a0f3-34e1bca5076d None None] MFA Rules not processed for user `4759f79e6a2a443db1bfe2c8764a850c`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:53:07.250269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e17ea2b3-81f7-4466-a0f3-34e1bca5076d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:07.308238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:53:07.336697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1152/2318] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:53:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:53:07.348006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758324a-0cc9-418f-ba94-ba62247388e0 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:07.349959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a758324a-0cc9-418f-ba94-ba62247388e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:07.445402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758324a-0cc9-418f-ba94-ba62247388e0 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:07.449222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff25348-ed59-4230-b892-82aa925ed60a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:07.449875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff25348-ed59-4230-b892-82aa925ed60a service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:07.450160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff25348-ed59-4230-b892-82aa925ed60a service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:07.450440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff25348-ed59-4230-b892-82aa925ed60a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:07.459389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff25348-ed59-4230-b892-82aa925ed60a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:07.460875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff25348-ed59-4230-b892-82aa925ed60a service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:07.470837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1167/2319] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:53:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:53:08.235260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5192d01-3644-455c-97a1-6daedfa9aa5a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:08.235753 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5192d01-3644-455c-97a1-6daedfa9aa5a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:08.235851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5192d01-3644-455c-97a1-6daedfa9aa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:08.254174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f5192d01-3644-455c-97a1-6daedfa9aa5a None None] MFA Rules not processed for user `04e35f840ef045f0b44fab3e57a1a3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:53:08.255355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f5192d01-3644-455c-97a1-6daedfa9aa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:08.312552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1153/2320] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:53:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:53:17.466110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:53:18.304973 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=34 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:53:21.406197 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6137-90f4-4ce7-bc4e-ca1a8e005532 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:21.406663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6137-90f4-4ce7-bc4e-ca1a8e005532 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:21.406747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed6137-90f4-4ce7-bc4e-ca1a8e005532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:21.429042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-15ed6137-90f4-4ce7-bc4e-ca1a8e005532 None None] MFA Rules not processed for user `5eb7e8b4befd4d519b440c632b59e724`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:53:21.432137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-15ed6137-90f4-4ce7-bc4e-ca1a8e005532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:21.504706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1168/2321] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:53:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:53:30.232012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feabd5-cff1-4ada-b588-1fb8a15d2942 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:30.232619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feabd5-cff1-4ada-b588-1fb8a15d2942 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:30.232619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34feabd5-cff1-4ada-b588-1fb8a15d2942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:30.252743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-34feabd5-cff1-4ada-b588-1fb8a15d2942 None None] MFA Rules not processed for user `2e78cf97bac4484f93c7359b86a26628`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:53:30.253940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-34feabd5-cff1-4ada-b588-1fb8a15d2942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:30.312967 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1154/2322] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:53:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:53:30.953087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed6137-90f4-4ce7-bc4e-ca1a8e005532 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:30.954621 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-15ed6137-90f4-4ce7-bc4e-ca1a8e005532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:31.029632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed6137-90f4-4ce7-bc4e-ca1a8e005532 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:31.033608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79259200-ff23-4b9e-811f-f8077110467b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:31.034298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79259200-ff23-4b9e-811f-f8077110467b service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:31.034701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79259200-ff23-4b9e-811f-f8077110467b service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:31.035010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79259200-ff23-4b9e-811f-f8077110467b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:31.037273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-79259200-ff23-4b9e-811f-f8077110467b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:31.125914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79259200-ff23-4b9e-811f-f8077110467b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:31.127461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79259200-ff23-4b9e-811f-f8077110467b service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:31.145970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1169/2323] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:53:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:53:35.755978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34feabd5-cff1-4ada-b588-1fb8a15d2942 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:35.757499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-34feabd5-cff1-4ada-b588-1fb8a15d2942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:35.773135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34feabd5-cff1-4ada-b588-1fb8a15d2942 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:35.777038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba81831-9de2-450c-8be2-9c5b6ddaed35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:35.777607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba81831-9de2-450c-8be2-9c5b6ddaed35 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:35.777891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba81831-9de2-450c-8be2-9c5b6ddaed35 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:35.778168 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba81831-9de2-450c-8be2-9c5b6ddaed35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:35.781502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1ba81831-9de2-450c-8be2-9c5b6ddaed35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:35.799715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba81831-9de2-450c-8be2-9c5b6ddaed35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:35.801354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba81831-9de2-450c-8be2-9c5b6ddaed35 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:35.816695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1155/2324] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:53:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:53:41.134278 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=79 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:53:42.829386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79259200-ff23-4b9e-811f-f8077110467b service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:42.892129 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79259200-ff23-4b9e-811f-f8077110467b service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:42.897004 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:42.897869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:42.898149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:42.898421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] PATH_INFO: `/v3/users/4759f79e6a2a443db1bfe2c8764a850c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:42.899609 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4759f79e6a2a443db1bfe2c8764a850c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:42.901564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:42.950826 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4759f79e6a2a443db1bfe2c8764a850c'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:42.963084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4759f79e6a2a443db1bfe2c8764a850c'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:42.963768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4759f79e6a2a443db1bfe2c8764a850c'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:42.964387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4759f79e6a2a443db1bfe2c8764a850c'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:42.972329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4759f79e6a2a443db1bfe2c8764a850c'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:42.985157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1170/2325] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:53:42 2022] DELETE /identity/v3/users/4759f79e6a2a443db1bfe2c8764a850c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:53:42.989656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d693e-f9ff-4379-b6ed-6fa97c31f36e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:42.990097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d693e-f9ff-4379-b6ed-6fa97c31f36e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:42.990210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d693e-f9ff-4379-b6ed-6fa97c31f36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:43.008137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d72d693e-f9ff-4379-b6ed-6fa97c31f36e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:53:43.009381 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d72d693e-f9ff-4379-b6ed-6fa97c31f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:43.069196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1156/2326] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:53:43.079900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:43.081428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:43.138328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb615efb-3d33-4808-910c-421f9eaede7d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:43.143264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae30196-a61c-445c-a862-db82e1415511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:43.143922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae30196-a61c-445c-a862-db82e1415511 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:43.144211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae30196-a61c-445c-a862-db82e1415511 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:43.144490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae30196-a61c-445c-a862-db82e1415511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:43.154068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae30196-a61c-445c-a862-db82e1415511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:43.156823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae30196-a61c-445c-a862-db82e1415511 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:43.166636 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1171/2327] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:53:43.496561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d693e-f9ff-4379-b6ed-6fa97c31f36e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:53:43.498117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d72d693e-f9ff-4379-b6ed-6fa97c31f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:53:43.513355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d693e-f9ff-4379-b6ed-6fa97c31f36e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:53:43.520135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:53:43.521130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:53:43.521521 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:53:43.521675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] PATH_INFO: `/v3/projects/38de3652d4e548faa55a8b0a860ab2e5` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:53:43.522932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38de3652d4e548faa55a8b0a860ab2e5)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:53:43.524903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:53:43.574994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38de3652d4e548faa55a8b0a860ab2e5', 'request_id': 'req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:43.583728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38de3652d4e548faa55a8b0a860ab2e5', 'request_id': 'req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:43.591510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38de3652d4e548faa55a8b0a860ab2e5', 'request_id': 'req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:43.602710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] The token cache is being invalidate because project 38de3652d4e548faa55a8b0a860ab2e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:53:43.603003 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:43.603625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:53:43.605230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1157/2328] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:53:43 2022] DELETE /identity/v3/projects/38de3652d4e548faa55a8b0a860ab2e5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:53:53.162138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:53:53.599655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:03.971078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae30196-a61c-445c-a862-db82e1415511 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:03.972821 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1ae30196-a61c-445c-a862-db82e1415511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:04.030551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae30196-a61c-445c-a862-db82e1415511 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:04.078793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:04.079444 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:04.079728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:04.080006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:04.081782 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:04.138780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:04.140240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:04.149334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1172/2329] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:08.078824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:08.080439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:08.163779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:08.166555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:08.176784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79cdb0-dcc6-401e-a82f-b3663f0f80d0 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:08.184246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fd845-bf37-40dd-94a4-64f78e65b742 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:08.184908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371ead5-b8ad-4951-88ad-59b45cb389ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:08.186348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371ead5-b8ad-4951-88ad-59b45cb389ab service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:08.186875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371ead5-b8ad-4951-88ad-59b45cb389ab service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:08.187452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371ead5-b8ad-4951-88ad-59b45cb389ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:08.189515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b371ead5-b8ad-4951-88ad-59b45cb389ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:08.192146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:08.193136 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:08.193653 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:08.194151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:08.199242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:08.276722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:08.279117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:08.284699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371ead5-b8ad-4951-88ad-59b45cb389ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:08.286630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371ead5-b8ad-4951-88ad-59b45cb389ab service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:08.298259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1158/2330] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:08.312763 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1173/2331] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:54:10.288813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52330b69-321b-41bf-86a1-64f64727935a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:10.289258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52330b69-321b-41bf-86a1-64f64727935a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:10.289444 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52330b69-321b-41bf-86a1-64f64727935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:10.308879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-52330b69-321b-41bf-86a1-64f64727935a None None] MFA Rules not processed for user `927ffbcd7078473f97f42bdb54acf57e`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:10.310090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-52330b69-321b-41bf-86a1-64f64727935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:10.318247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1159/2332] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:10.628238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:10.629696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:10.640133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:10.691427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0dcd0-1eb4-48d8-857e-0f3f7cc83a1b service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:10.695052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa0499-402c-480e-b639-94ad134437ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:10.695695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa0499-402c-480e-b639-94ad134437ab service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:10.695977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa0499-402c-480e-b639-94ad134437ab service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:10.696251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa0499-402c-480e-b639-94ad134437ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:10.698048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e3fa0499-402c-480e-b639-94ad134437ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:10.707459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa0499-402c-480e-b639-94ad134437ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:10.708968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa0499-402c-480e-b639-94ad134437ab service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:10.716786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1174/2333] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:11.030754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52330b69-321b-41bf-86a1-64f64727935a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:11.032157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-52330b69-321b-41bf-86a1-64f64727935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:11.107303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52330b69-321b-41bf-86a1-64f64727935a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:11.112319 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:11.112987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:11.113269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:11.113544 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:11.115306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:11.126902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:11.128462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:11.138361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1160/2334] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:15.755541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa0499-402c-480e-b639-94ad134437ab service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:15.774014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa0499-402c-480e-b639-94ad134437ab service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:15.781324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:15.782005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:15.782911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:15.782911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:15.784523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:15.800845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:15.802595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:15.815395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1175/2335] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:15.925798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:15.927681 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:15.943344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadfb4c-fdb8-4f61-8dba-6e54f98d4596 service placement] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:15.956282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f5ddb-e13d-472b-84df-531c4e63485a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:15.957008 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f5ddb-e13d-472b-84df-531c4e63485a service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:15.957291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f5ddb-e13d-472b-84df-531c4e63485a service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:15.957655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f5ddb-e13d-472b-84df-531c4e63485a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:15.964563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-795f5ddb-e13d-472b-84df-531c4e63485a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:15.981564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f5ddb-e13d-472b-84df-531c4e63485a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:15.983754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f5ddb-e13d-472b-84df-531c4e63485a service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:15.993883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1161/2336] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:21.875908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:21.877362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:21.894578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beefc5c-27d8-430d-9c07-492436f59088 service nova] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:21.900149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239484c2-d1ef-4f5a-99c4-d45887d4df74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:21.900831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239484c2-d1ef-4f5a-99c4-d45887d4df74 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:21.901114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239484c2-d1ef-4f5a-99c4-d45887d4df74 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:21.901391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239484c2-d1ef-4f5a-99c4-d45887d4df74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:21.903139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-239484c2-d1ef-4f5a-99c4-d45887d4df74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:21.915518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239484c2-d1ef-4f5a-99c4-d45887d4df74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:21.917163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239484c2-d1ef-4f5a-99c4-d45887d4df74 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:21.928743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1176/2337] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:24.859321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f5ddb-e13d-472b-84df-531c4e63485a service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:24.868255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f5ddb-e13d-472b-84df-531c4e63485a service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:24.871925 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce34345-42c3-4bf6-b566-519cac4b9ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:24.872645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce34345-42c3-4bf6-b566-519cac4b9ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:24.872930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce34345-42c3-4bf6-b566-519cac4b9ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:24.873205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce34345-42c3-4bf6-b566-519cac4b9ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:24.874971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-bce34345-42c3-4bf6-b566-519cac4b9ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:24.881169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:24.885680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce34345-42c3-4bf6-b566-519cac4b9ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:24.887083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce34345-42c3-4bf6-b566-519cac4b9ec2 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:24.895906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1162/2338] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:25.000203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207411a-0220-47b3-aa81-87d24491f26b None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:25.000797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207411a-0220-47b3-aa81-87d24491f26b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:25.000797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207411a-0220-47b3-aa81-87d24491f26b None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:25.002176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1177/2339] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:54:25.008305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657f923-e45b-40f1-a852-a27fcc21990d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:25.008699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657f923-e45b-40f1-a852-a27fcc21990d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:25.009055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657f923-e45b-40f1-a852-a27fcc21990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:25.028207 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-b657f923-e45b-40f1-a852-a27fcc21990d None None] MFA Rules not processed for user `e38a311f2b614a3598f6b51b3a235178`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:25.030355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b657f923-e45b-40f1-a852-a27fcc21990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:25.037097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1163/2340] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:54:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:25.053625 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207411a-0220-47b3-aa81-87d24491f26b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:25.069845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207411a-0220-47b3-aa81-87d24491f26b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:25.074527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6deb9-00a0-4e70-a595-abd1e8fec016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:25.075176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6deb9-00a0-4e70-a595-abd1e8fec016 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:25.076234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6deb9-00a0-4e70-a595-abd1e8fec016 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:25.076747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6deb9-00a0-4e70-a595-abd1e8fec016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:25.086426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6deb9-00a0-4e70-a595-abd1e8fec016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:25.087525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6deb9-00a0-4e70-a595-abd1e8fec016 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:25.098585 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1178/2341] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:54:25.219219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781938dc-5bd0-4774-9880-9d6ce8b15ea5 None None] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:25.219523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781938dc-5bd0-4774-9880-9d6ce8b15ea5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:25.219799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781938dc-5bd0-4774-9880-9d6ce8b15ea5 None None] PATH_INFO: `/` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:25.221391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1164/2342] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:54:25.227900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6329f1-5b55-415f-863d-bd55c07f2b3a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:25.228187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6329f1-5b55-415f-863d-bd55c07f2b3a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:25.228689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6329f1-5b55-415f-863d-bd55c07f2b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:25.248893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9f6329f1-5b55-415f-863d-bd55c07f2b3a None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:25.250217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9f6329f1-5b55-415f-863d-bd55c07f2b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:25.267358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1179/2343] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:54:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:25.278442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781938dc-5bd0-4774-9880-9d6ce8b15ea5 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:25.289926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781938dc-5bd0-4774-9880-9d6ce8b15ea5 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:25.293552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2360d1-fd25-4bad-ac49-8e26d176b15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:25.294236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2360d1-fd25-4bad-ac49-8e26d176b15e service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:25.294692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2360d1-fd25-4bad-ac49-8e26d176b15e service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:25.294692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2360d1-fd25-4bad-ac49-8e26d176b15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:25.305461 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2360d1-fd25-4bad-ac49-8e26d176b15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:25.306895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2360d1-fd25-4bad-ac49-8e26d176b15e service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:25.318713 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1165/2344] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:26.724457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6329f1-5b55-415f-863d-bd55c07f2b3a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:26.739271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6329f1-5b55-415f-863d-bd55c07f2b3a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:26.749296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fdf91-0c5d-4afc-9b61-5d386f3f4890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:26.749965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fdf91-0c5d-4afc-9b61-5d386f3f4890 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:26.750243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fdf91-0c5d-4afc-9b61-5d386f3f4890 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:26.750514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fdf91-0c5d-4afc-9b61-5d386f3f4890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:26.753010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-852fdf91-0c5d-4afc-9b61-5d386f3f4890 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:26.767569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fdf91-0c5d-4afc-9b61-5d386f3f4890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:26.769798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fdf91-0c5d-4afc-9b61-5d386f3f4890 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:26.782687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1180/2345] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:28.052264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2360d1-fd25-4bad-ac49-8e26d176b15e service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:28.065161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2360d1-fd25-4bad-ac49-8e26d176b15e service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:28.070877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77eb2a-aa65-497f-ab71-3add227d71ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:28.071525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77eb2a-aa65-497f-ab71-3add227d71ad service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:28.071810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77eb2a-aa65-497f-ab71-3add227d71ad service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:28.072090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77eb2a-aa65-497f-ab71-3add227d71ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:28.074018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-dc77eb2a-aa65-497f-ab71-3add227d71ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:28.136277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77eb2a-aa65-497f-ab71-3add227d71ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:28.138032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77eb2a-aa65-497f-ab71-3add227d71ad service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:28.147854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1166/2346] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:32.593339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fdf91-0c5d-4afc-9b61-5d386f3f4890 service nova] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:32.602658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fdf91-0c5d-4afc-9b61-5d386f3f4890 service nova] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:32.606650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:32.607431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:32.607715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:32.607989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:32.609936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:32.682109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:32.683584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:32.704644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1181/2347] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:33.693726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea1b5d-0c37-4d5a-aaef-2c11783a3ff6 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:33.694228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea1b5d-0c37-4d5a-aaef-2c11783a3ff6 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:33.694396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea1b5d-0c37-4d5a-aaef-2c11783a3ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:33.714524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-cdea1b5d-0c37-4d5a-aaef-2c11783a3ff6 None None] MFA Rules not processed for user `04e35f840ef045f0b44fab3e57a1a3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:33.716457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-cdea1b5d-0c37-4d5a-aaef-2c11783a3ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:33.725403 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1167/2348] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:37.065574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:37.069545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:37.086312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a208-7453-41ce-a019-2666cc83521d service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:37.091827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:37.093235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:37.093654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:37.094074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] PATH_INFO: `/v3/users/5eb7e8b4befd4d519b440c632b59e724` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:37.095531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eb7e8b4befd4d519b440c632b59e724)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:37.097512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:37.145698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:37.152167 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eb7e8b4befd4d519b440c632b59e724'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.167121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eb7e8b4befd4d519b440c632b59e724'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.167861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eb7e8b4befd4d519b440c632b59e724'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.168482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eb7e8b4befd4d519b440c632b59e724'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.177395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a4acd085-021b-4491-b917-561ebc99798e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eb7e8b4befd4d519b440c632b59e724'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.197685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1182/2349] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:54:37 2022] DELETE /identity/v3/users/5eb7e8b4befd4d519b440c632b59e724 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:37.387029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea1b5d-0c37-4d5a-aaef-2c11783a3ff6 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:37.401271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea1b5d-0c37-4d5a-aaef-2c11783a3ff6 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:37.485114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:37.486116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:37.486468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:37.486810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] PATH_INFO: `/v3/projects/7ee4f2aaeed54d0fb16474b971f333d6` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:37.488063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee4f2aaeed54d0fb16474b971f333d6)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:37.490250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:37.548235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee4f2aaeed54d0fb16474b971f333d6', 'request_id': 'req-99f763d8-894e-4b8b-823a-97c2702f11da'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.556230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee4f2aaeed54d0fb16474b971f333d6', 'request_id': 'req-99f763d8-894e-4b8b-823a-97c2702f11da'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.567409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee4f2aaeed54d0fb16474b971f333d6', 'request_id': 'req-99f763d8-894e-4b8b-823a-97c2702f11da'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.572940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:37.582050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] The token cache is being invalidate because project 7ee4f2aaeed54d0fb16474b971f333d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:54:37.582535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.583657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-99f763d8-894e-4b8b-823a-97c2702f11da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:37.585419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1168/2350] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:54:37 2022] DELETE /identity/v3/projects/7ee4f2aaeed54d0fb16474b971f333d6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:37.590947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd05d8-8498-4b12-a2c5-7da3a9279d48 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:37.592220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd05d8-8498-4b12-a2c5-7da3a9279d48 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:37.592555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd05d8-8498-4b12-a2c5-7da3a9279d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:37.616257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0dbd05d8-8498-4b12-a2c5-7da3a9279d48 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:37.617542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0dbd05d8-8498-4b12-a2c5-7da3a9279d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:37.683618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1183/2351] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:37.691174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f79a8d-a9d8-48c2-a360-2824aef12644 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:37.691464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f79a8d-a9d8-48c2-a360-2824aef12644 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:37.691744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f79a8d-a9d8-48c2-a360-2824aef12644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:37.712620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-07f79a8d-a9d8-48c2-a360-2824aef12644 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:37.714080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-07f79a8d-a9d8-48c2-a360-2824aef12644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:37.722929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1169/2352] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:54:37.733370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd05d8-8498-4b12-a2c5-7da3a9279d48 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:37.749749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd05d8-8498-4b12-a2c5-7da3a9279d48 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:37.754986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fae43e-98b3-4566-a4aa-f947bfbc55a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:37.755718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fae43e-98b3-4566-a4aa-f947bfbc55a7 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:37.755998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fae43e-98b3-4566-a4aa-f947bfbc55a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:37.756273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fae43e-98b3-4566-a4aa-f947bfbc55a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:37.757031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fae43e-98b3-4566-a4aa-f947bfbc55a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:54:37.757342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fae43e-98b3-4566-a4aa-f947bfbc55a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:37.759105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fae43e-98b3-4566-a4aa-f947bfbc55a7 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:37.771471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1184/2353] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:54:37.775704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f79a8d-a9d8-48c2-a360-2824aef12644 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:37.785195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f79a8d-a9d8-48c2-a360-2824aef12644 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:37.789871 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85948f26-d4fd-434d-826b-896f4a8d808d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:37.790518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85948f26-d4fd-434d-826b-896f4a8d808d admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:37.790798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85948f26-d4fd-434d-826b-896f4a8d808d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:37.791073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85948f26-d4fd-434d-826b-896f4a8d808d admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:37.791761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85948f26-d4fd-434d-826b-896f4a8d808d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:37.794293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85948f26-d4fd-434d-826b-896f4a8d808d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:37.823816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1170/2354] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:54:37 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:54:37.825153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fae43e-98b3-4566-a4aa-f947bfbc55a7 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:37.835554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fae43e-98b3-4566-a4aa-f947bfbc55a7 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:37.905612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc733e-d07f-4222-ab63-6ee306301804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:37.906080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc733e-d07f-4222-ab63-6ee306301804 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:37.914556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc733e-d07f-4222-ab63-6ee306301804 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:37.914886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc733e-d07f-4222-ab63-6ee306301804 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:37.915696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc733e-d07f-4222-ab63-6ee306301804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:37.917779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc733e-d07f-4222-ab63-6ee306301804 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:37.949128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1185/2355] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:54:37 2022] POST /identity/v3/users => generated 302 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:54:37.952198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85948f26-d4fd-434d-826b-896f4a8d808d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:37.965561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85948f26-d4fd-434d-826b-896f4a8d808d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:37.975333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33ad07-4395-4328-846f-0b22fa4abf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:37.976209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33ad07-4395-4328-846f-0b22fa4abf0c admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:37.976679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33ad07-4395-4328-846f-0b22fa4abf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:37.977120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33ad07-4395-4328-846f-0b22fa4abf0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:37.978555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33ad07-4395-4328-846f-0b22fa4abf0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:37.980924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33ad07-4395-4328-846f-0b22fa4abf0c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:37.999567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1171/2356] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:54:38.001350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc733e-d07f-4222-ab63-6ee306301804 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:38.017206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc733e-d07f-4222-ab63-6ee306301804 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:38.033493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a9f1a-74cf-4dc0-becf-87cb91594553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:38.034238 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a9f1a-74cf-4dc0-becf-87cb91594553 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:38.034520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a9f1a-74cf-4dc0-becf-87cb91594553 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:38.034796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a9f1a-74cf-4dc0-becf-87cb91594553 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:38.035592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a9f1a-74cf-4dc0-becf-87cb91594553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:38.037315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a9f1a-74cf-4dc0-becf-87cb91594553 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:38.054439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1186/2357] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:54:38.058172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33ad07-4395-4328-846f-0b22fa4abf0c admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:38.069971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33ad07-4395-4328-846f-0b22fa4abf0c admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:38.076734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ab3c4-a23c-4231-9f30-6a2f70d6e491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:38.078497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ab3c4-a23c-4231-9f30-6a2f70d6e491 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:38.079028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ab3c4-a23c-4231-9f30-6a2f70d6e491 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:38.079522 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ab3c4-a23c-4231-9f30-6a2f70d6e491 admin admin] PATH_INFO: `/v3/projects/c1649d5fb3ea40bba8fb383c270a007b/users/643a394c14bc4719aeec35a259b12bb5/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:38.120167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ab3c4-a23c-4231-9f30-6a2f70d6e491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1649d5fb3ea40bba8fb383c270a007b, user_id=643a394c14bc4719aeec35a259b12bb5, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:38.124980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ab3c4-a23c-4231-9f30-6a2f70d6e491 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:38.150677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1172/2358] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:54:38 2022] PUT /identity/v3/projects/c1649d5fb3ea40bba8fb383c270a007b/users/643a394c14bc4719aeec35a259b12bb5/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:38.156380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f14995-eede-442a-a5d3-80a536bb64be None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:38.157008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f14995-eede-442a-a5d3-80a536bb64be None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:38.157434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f14995-eede-442a-a5d3-80a536bb64be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:38.189696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-a2f14995-eede-442a-a5d3-80a536bb64be None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:38.191143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a2f14995-eede-442a-a5d3-80a536bb64be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:38.216746 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:38.275229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1187/2359] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:38.288873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ab3c4-a23c-4231-9f30-6a2f70d6e491 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:38.290634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-433ab3c4-a23c-4231-9f30-6a2f70d6e491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:38.376826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ab3c4-a23c-4231-9f30-6a2f70d6e491 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:38.382376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347e8bc-d61a-4886-94a7-c3dec8517546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:38.383079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347e8bc-d61a-4886-94a7-c3dec8517546 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:38.383360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347e8bc-d61a-4886-94a7-c3dec8517546 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:38.383636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347e8bc-d61a-4886-94a7-c3dec8517546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:38.395095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347e8bc-d61a-4886-94a7-c3dec8517546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:38.396668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347e8bc-d61a-4886-94a7-c3dec8517546 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:38.407914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1173/2360] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:39.149594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6ffc4-18e9-49b4-8d97-080e43f1b4c9 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:39.149934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6ffc4-18e9-49b4-8d97-080e43f1b4c9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:39.150215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6ffc4-18e9-49b4-8d97-080e43f1b4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:39.180677 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-66c6ffc4-18e9-49b4-8d97-080e43f1b4c9 None None] MFA Rules not processed for user `643a394c14bc4719aeec35a259b12bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:39.182265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-66c6ffc4-18e9-49b4-8d97-080e43f1b4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:39.251352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1188/2361] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:39.255485 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347e8bc-d61a-4886-94a7-c3dec8517546 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:39.266406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347e8bc-d61a-4886-94a7-c3dec8517546 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:39.272343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0999416-6a86-48af-b5c4-4532e8b20556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:39.273170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0999416-6a86-48af-b5c4-4532e8b20556 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:39.273538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0999416-6a86-48af-b5c4-4532e8b20556 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:39.273900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0999416-6a86-48af-b5c4-4532e8b20556 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:39.274690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0999416-6a86-48af-b5c4-4532e8b20556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:39.277624 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0999416-6a86-48af-b5c4-4532e8b20556 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:39.318634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1174/2362] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:54:39 2022] POST /identity/v3/projects => generated 403 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:54:39.323160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6ffc4-18e9-49b4-8d97-080e43f1b4c9 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:39.337421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6ffc4-18e9-49b4-8d97-080e43f1b4c9 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:39.414802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffaf30-3ed3-4800-8525-fd9c1447a0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:39.415496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffaf30-3ed3-4800-8525-fd9c1447a0ca admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:39.415783 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffaf30-3ed3-4800-8525-fd9c1447a0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:39.416072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffaf30-3ed3-4800-8525-fd9c1447a0ca admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:39.416808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffaf30-3ed3-4800-8525-fd9c1447a0ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:39.418935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffaf30-3ed3-4800-8525-fd9c1447a0ca admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:39.459868 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1189/2363] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:54:39 2022] POST /identity/v3/users => generated 308 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:54:39.459978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0999416-6a86-48af-b5c4-4532e8b20556 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:39.474532 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0999416-6a86-48af-b5c4-4532e8b20556 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:39.480526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a176f0-368e-48df-8e0b-4cdda6e39c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:39.481496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a176f0-368e-48df-8e0b-4cdda6e39c7a admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:39.481976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a176f0-368e-48df-8e0b-4cdda6e39c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:39.482263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a176f0-368e-48df-8e0b-4cdda6e39c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:39.483103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a176f0-368e-48df-8e0b-4cdda6e39c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:39.484934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a176f0-368e-48df-8e0b-4cdda6e39c7a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:39.505155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1175/2364] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:54:39.506767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffaf30-3ed3-4800-8525-fd9c1447a0ca admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:39.519197 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffaf30-3ed3-4800-8525-fd9c1447a0ca admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:39.524316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6d7bc-8bdf-4474-a967-65033a87eae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:39.525250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6d7bc-8bdf-4474-a967-65033a87eae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:39.525539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6d7bc-8bdf-4474-a967-65033a87eae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:39.525811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6d7bc-8bdf-4474-a967-65033a87eae7 admin admin] PATH_INFO: `/v3/projects/b8e6693754f54c37a7d67392febe9e1b/users/dc7ba38007d241be9ea7eb2d921798a7/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:39.544919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6d7bc-8bdf-4474-a967-65033a87eae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e6693754f54c37a7d67392febe9e1b, user_id=dc7ba38007d241be9ea7eb2d921798a7, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:39.547571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6d7bc-8bdf-4474-a967-65033a87eae7 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:39.563274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1190/2365] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:54:39 2022] PUT /identity/v3/projects/b8e6693754f54c37a7d67392febe9e1b/users/dc7ba38007d241be9ea7eb2d921798a7/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:39.567462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a176f0-368e-48df-8e0b-4cdda6e39c7a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:39.578476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a176f0-368e-48df-8e0b-4cdda6e39c7a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:39.645638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc5abc-c929-4493-ac2f-33d30a3710f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:39.646332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc5abc-c929-4493-ac2f-33d30a3710f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:39.646618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc5abc-c929-4493-ac2f-33d30a3710f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:39.646898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc5abc-c929-4493-ac2f-33d30a3710f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:39.647636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc5abc-c929-4493-ac2f-33d30a3710f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:39.649303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc5abc-c929-4493-ac2f-33d30a3710f0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:39.653460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:39.670403 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1176/2366] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:54:39.671612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6d7bc-8bdf-4474-a967-65033a87eae7 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:39.682243 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6d7bc-8bdf-4474-a967-65033a87eae7 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:39.687813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a715b40-dbfb-4fdb-a076-b0d03f5514d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:39.689040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a715b40-dbfb-4fdb-a076-b0d03f5514d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:39.689327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a715b40-dbfb-4fdb-a076-b0d03f5514d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:39.689655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a715b40-dbfb-4fdb-a076-b0d03f5514d2 admin admin] PATH_INFO: `/v3/projects/b8e6693754f54c37a7d67392febe9e1b/users/dc7ba38007d241be9ea7eb2d921798a7/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:39.691662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a715b40-dbfb-4fdb-a076-b0d03f5514d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e6693754f54c37a7d67392febe9e1b, user_id=dc7ba38007d241be9ea7eb2d921798a7, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:39.695595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a715b40-dbfb-4fdb-a076-b0d03f5514d2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:39.711522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1191/2367] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:54:39 2022] PUT /identity/v3/projects/b8e6693754f54c37a7d67392febe9e1b/users/dc7ba38007d241be9ea7eb2d921798a7/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:40.682980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b019b31-1cd7-49f9-a97d-1e5dac05904b None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:40.683505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b019b31-1cd7-49f9-a97d-1e5dac05904b None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:40.683616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b019b31-1cd7-49f9-a97d-1e5dac05904b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:40.722088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0b019b31-1cd7-49f9-a97d-1e5dac05904b None None] MFA Rules not processed for user `dc7ba38007d241be9ea7eb2d921798a7`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:40.723368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0b019b31-1cd7-49f9-a97d-1e5dac05904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:40.817889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1177/2368] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:42.005953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a715b40-dbfb-4fdb-a076-b0d03f5514d2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:42.021847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:42.091822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a715b40-dbfb-4fdb-a076-b0d03f5514d2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:42.097106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:42.097995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:42.098277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:42.098552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] PATH_INFO: `/v3/users/643a394c14bc4719aeec35a259b12bb5` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:42.099824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643a394c14bc4719aeec35a259b12bb5)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:42.102031 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:42.167156 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643a394c14bc4719aeec35a259b12bb5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.182780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643a394c14bc4719aeec35a259b12bb5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.183524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643a394c14bc4719aeec35a259b12bb5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.184251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643a394c14bc4719aeec35a259b12bb5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.191571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643a394c14bc4719aeec35a259b12bb5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.204960 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1192/2369] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:54:42 2022] DELETE /identity/v3/users/643a394c14bc4719aeec35a259b12bb5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:42.449437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b019b31-1cd7-49f9-a97d-1e5dac05904b None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:42.464416 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b019b31-1cd7-49f9-a97d-1e5dac05904b None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:42.544109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:42.545400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:42.546333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:42.546620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] PATH_INFO: `/v3/projects/c1649d5fb3ea40bba8fb383c270a007b` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:42.547872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1649d5fb3ea40bba8fb383c270a007b)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:42.551102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:42.601379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1649d5fb3ea40bba8fb383c270a007b', 'request_id': 'req-6c492997-e225-497d-94c8-46dae7da5bde'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.608681 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1649d5fb3ea40bba8fb383c270a007b', 'request_id': 'req-6c492997-e225-497d-94c8-46dae7da5bde'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.616211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1649d5fb3ea40bba8fb383c270a007b', 'request_id': 'req-6c492997-e225-497d-94c8-46dae7da5bde'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.627001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] The token cache is being invalidate because project c1649d5fb3ea40bba8fb383c270a007b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:54:42.627298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.627935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.629516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1178/2370] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:54:42 2022] DELETE /identity/v3/projects/c1649d5fb3ea40bba8fb383c270a007b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:42.633425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:42.634847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:42.648022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef758-9f05-42a7-854a-f758b95b4623 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:42.702511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:42.703373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:42.703651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:42.703925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] PATH_INFO: `/v3/users/dc7ba38007d241be9ea7eb2d921798a7` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:42.705131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc7ba38007d241be9ea7eb2d921798a7)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:42.707036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:42.735539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:42.755212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc7ba38007d241be9ea7eb2d921798a7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.767387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc7ba38007d241be9ea7eb2d921798a7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.768043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc7ba38007d241be9ea7eb2d921798a7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.768690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc7ba38007d241be9ea7eb2d921798a7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.776114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ca59e136-9b43-41b5-a077-614f1505d27e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc7ba38007d241be9ea7eb2d921798a7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:42.788970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1193/2371] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:54:42 2022] DELETE /identity/v3/users/dc7ba38007d241be9ea7eb2d921798a7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:42.960282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:42.961841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:42.976440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c492997-e225-497d-94c8-46dae7da5bde admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:42.986082 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:43.035977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.036884 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.037188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.037467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] PATH_INFO: `/v3/projects/b8e6693754f54c37a7d67392febe9e1b` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.038645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e6693754f54c37a7d67392febe9e1b)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.040539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.089425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e6693754f54c37a7d67392febe9e1b', 'request_id': 'req-2c99994a-69de-4160-81d9-0fb1e6ec7d16'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:43.096716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e6693754f54c37a7d67392febe9e1b', 'request_id': 'req-2c99994a-69de-4160-81d9-0fb1e6ec7d16'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:43.107542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e6693754f54c37a7d67392febe9e1b', 'request_id': 'req-2c99994a-69de-4160-81d9-0fb1e6ec7d16'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:43.119315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] The token cache is being invalidate because project b8e6693754f54c37a7d67392febe9e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:54:43.119605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:43.120347 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2c99994a-69de-4160-81d9-0fb1e6ec7d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:54:43.123415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1179/2372] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:54:42 2022] DELETE /identity/v3/projects/b8e6693754f54c37a7d67392febe9e1b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:43.128373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8d341-777f-406c-9e4b-772c18c001fc None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.128938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8d341-777f-406c-9e4b-772c18c001fc None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.129112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8d341-777f-406c-9e4b-772c18c001fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.149055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-67e8d341-777f-406c-9e4b-772c18c001fc None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:43.150310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-67e8d341-777f-406c-9e4b-772c18c001fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:43.224390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1194/2373] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:43.232698 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85affb3e-7a4a-40e6-becc-43c3a8e96080 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.232990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85affb3e-7a4a-40e6-becc-43c3a8e96080 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.233278 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85affb3e-7a4a-40e6-becc-43c3a8e96080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.254013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-85affb3e-7a4a-40e6-becc-43c3a8e96080 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:43.255223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-85affb3e-7a4a-40e6-becc-43c3a8e96080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:43.264367 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1180/2374] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:43.268533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8d341-777f-406c-9e4b-772c18c001fc None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.281939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8d341-777f-406c-9e4b-772c18c001fc None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.286629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3b59b-ddf6-4f80-a089-cf611b6cd54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.287291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3b59b-ddf6-4f80-a089-cf611b6cd54d admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.287575 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3b59b-ddf6-4f80-a089-cf611b6cd54d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.287852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3b59b-ddf6-4f80-a089-cf611b6cd54d admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.288607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3b59b-ddf6-4f80-a089-cf611b6cd54d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:54:43.288925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3b59b-ddf6-4f80-a089-cf611b6cd54d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.290531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3b59b-ddf6-4f80-a089-cf611b6cd54d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.302393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1195/2375] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:54:43.306620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85affb3e-7a4a-40e6-becc-43c3a8e96080 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.316516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85affb3e-7a4a-40e6-becc-43c3a8e96080 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.321279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e082152-b1f7-4746-a811-72fa8405a815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.321987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e082152-b1f7-4746-a811-72fa8405a815 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.322279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e082152-b1f7-4746-a811-72fa8405a815 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.322554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e082152-b1f7-4746-a811-72fa8405a815 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.323251 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e082152-b1f7-4746-a811-72fa8405a815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.326100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e082152-b1f7-4746-a811-72fa8405a815 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.351196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1181/2376] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:54:43 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:54:43.355359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3b59b-ddf6-4f80-a089-cf611b6cd54d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.365482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3b59b-ddf6-4f80-a089-cf611b6cd54d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.423863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3235e7-f4ea-4c9f-a063-77fd8cae60b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.424542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3235e7-f4ea-4c9f-a063-77fd8cae60b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.424868 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3235e7-f4ea-4c9f-a063-77fd8cae60b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.425140 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3235e7-f4ea-4c9f-a063-77fd8cae60b7 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.426056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3235e7-f4ea-4c9f-a063-77fd8cae60b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.428623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3235e7-f4ea-4c9f-a063-77fd8cae60b7 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.449318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1196/2377] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:54:43 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:54:43.454351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e082152-b1f7-4746-a811-72fa8405a815 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.464800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e082152-b1f7-4746-a811-72fa8405a815 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.469898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d624d-0de6-4f24-a4ba-bcd812fc0b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.470560 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d624d-0de6-4f24-a4ba-bcd812fc0b4e admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.470839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d624d-0de6-4f24-a4ba-bcd812fc0b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.471114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d624d-0de6-4f24-a4ba-bcd812fc0b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.471838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d624d-0de6-4f24-a4ba-bcd812fc0b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.473469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d624d-0de6-4f24-a4ba-bcd812fc0b4e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.484044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1182/2378] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:54:43.487859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3235e7-f4ea-4c9f-a063-77fd8cae60b7 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.499555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3235e7-f4ea-4c9f-a063-77fd8cae60b7 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.504408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bee087-2a8c-4999-9d47-8ef5b9681db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.505111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee087-2a8c-4999-9d47-8ef5b9681db3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.505431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee087-2a8c-4999-9d47-8ef5b9681db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.505727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee087-2a8c-4999-9d47-8ef5b9681db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.506452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bee087-2a8c-4999-9d47-8ef5b9681db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.508052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bee087-2a8c-4999-9d47-8ef5b9681db3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.523020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1197/2379] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:54:43.526783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d624d-0de6-4f24-a4ba-bcd812fc0b4e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.537154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d624d-0de6-4f24-a4ba-bcd812fc0b4e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.541735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406396-c378-460b-872e-a4c5eac13cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.542621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406396-c378-460b-872e-a4c5eac13cde admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.542900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406396-c378-460b-872e-a4c5eac13cde admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.543179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406396-c378-460b-872e-a4c5eac13cde admin admin] PATH_INFO: `/v3/projects/2d3ca8443d7645cbb459b8c7656ac324/users/28456b998d2d4f7f847f22ad92082362/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.558328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa406396-c378-460b-872e-a4c5eac13cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d3ca8443d7645cbb459b8c7656ac324, user_id=28456b998d2d4f7f847f22ad92082362, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.562123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa406396-c378-460b-872e-a4c5eac13cde admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.575636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1183/2380] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:54:43 2022] PUT /identity/v3/projects/2d3ca8443d7645cbb459b8c7656ac324/users/28456b998d2d4f7f847f22ad92082362/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:43.586814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71b419-601b-4ec3-b970-83715d2998e4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.587105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71b419-601b-4ec3-b970-83715d2998e4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.587388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71b419-601b-4ec3-b970-83715d2998e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.606915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:43.615110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-fc71b419-601b-4ec3-b970-83715d2998e4 None None] MFA Rules not processed for user `28456b998d2d4f7f847f22ad92082362`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:43.616814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-fc71b419-601b-4ec3-b970-83715d2998e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:43.686931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1198/2381] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:43.691599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406396-c378-460b-872e-a4c5eac13cde admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.735670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:43.758679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406396-c378-460b-872e-a4c5eac13cde admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.763457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474cc48-9735-4801-b721-1d2a2f94e997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.764167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474cc48-9735-4801-b721-1d2a2f94e997 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.764439 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474cc48-9735-4801-b721-1d2a2f94e997 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.764754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474cc48-9735-4801-b721-1d2a2f94e997 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.765464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474cc48-9735-4801-b721-1d2a2f94e997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.767375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474cc48-9735-4801-b721-1d2a2f94e997 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.799185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1184/2382] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:54:43 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:54:43.803306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71b419-601b-4ec3-b970-83715d2998e4 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.815393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71b419-601b-4ec3-b970-83715d2998e4 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.877464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c3fff-3774-4212-971e-d173c2388562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.878076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c3fff-3774-4212-971e-d173c2388562 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.878360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c3fff-3774-4212-971e-d173c2388562 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.878641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c3fff-3774-4212-971e-d173c2388562 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.879349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c3fff-3774-4212-971e-d173c2388562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.881622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c3fff-3774-4212-971e-d173c2388562 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.903768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1199/2383] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:54:43 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:54:43.906216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474cc48-9735-4801-b721-1d2a2f94e997 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.917222 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474cc48-9735-4801-b721-1d2a2f94e997 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.923923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c9ece-1916-4336-9a65-3dcebf4bfcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.924262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c9ece-1916-4336-9a65-3dcebf4bfcee admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.924546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c9ece-1916-4336-9a65-3dcebf4bfcee admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.924854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c9ece-1916-4336-9a65-3dcebf4bfcee admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.925595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c9ece-1916-4336-9a65-3dcebf4bfcee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:43.927183 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c9ece-1916-4336-9a65-3dcebf4bfcee admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:43.948620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1185/2384] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:54:43.952969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c3fff-3774-4212-971e-d173c2388562 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:43.965995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c3fff-3774-4212-971e-d173c2388562 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:43.972306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a729d-4ba8-4307-b211-de1449921ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:43.973240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a729d-4ba8-4307-b211-de1449921ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:43.973524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a729d-4ba8-4307-b211-de1449921ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:43.973801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a729d-4ba8-4307-b211-de1449921ec5 admin admin] PATH_INFO: `/v3/projects/39072db4d26940b58c766f47abd0dbe5/users/26725b170a2b429d91e91d4cafb6ec75/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:43.998711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a729d-4ba8-4307-b211-de1449921ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39072db4d26940b58c766f47abd0dbe5, user_id=26725b170a2b429d91e91d4cafb6ec75, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:44.001439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a729d-4ba8-4307-b211-de1449921ec5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:44.017355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1200/2385] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:54:43 2022] PUT /identity/v3/projects/39072db4d26940b58c766f47abd0dbe5/users/26725b170a2b429d91e91d4cafb6ec75/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:44.021957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c9ece-1916-4336-9a65-3dcebf4bfcee admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:44.033756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c9ece-1916-4336-9a65-3dcebf4bfcee admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:44.094665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507c67e-2a41-4758-935d-6691f9afb144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:44.095334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507c67e-2a41-4758-935d-6691f9afb144 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:44.095619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507c67e-2a41-4758-935d-6691f9afb144 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:44.095895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507c67e-2a41-4758-935d-6691f9afb144 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:44.096657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507c67e-2a41-4758-935d-6691f9afb144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:44.098249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507c67e-2a41-4758-935d-6691f9afb144 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:44.107909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1186/2386] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:54:44.111993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a729d-4ba8-4307-b211-de1449921ec5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:44.123163 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a729d-4ba8-4307-b211-de1449921ec5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:44.127855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337c9d9-c9de-4784-8770-4d455cbfef1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:44.128706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337c9d9-c9de-4784-8770-4d455cbfef1e admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:44.129289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337c9d9-c9de-4784-8770-4d455cbfef1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:44.129289 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337c9d9-c9de-4784-8770-4d455cbfef1e admin admin] PATH_INFO: `/v3/projects/39072db4d26940b58c766f47abd0dbe5/users/26725b170a2b429d91e91d4cafb6ec75/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:44.131230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337c9d9-c9de-4784-8770-4d455cbfef1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=39072db4d26940b58c766f47abd0dbe5, user_id=26725b170a2b429d91e91d4cafb6ec75, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:44.133920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337c9d9-c9de-4784-8770-4d455cbfef1e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:44.145304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1201/2387] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:54:44 2022] PUT /identity/v3/projects/39072db4d26940b58c766f47abd0dbe5/users/26725b170a2b429d91e91d4cafb6ec75/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:54:44.155367 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c20652-666a-4865-8abc-a56d7e6c1554 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:44.155659 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c20652-666a-4865-8abc-a56d7e6c1554 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:44.155944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c20652-666a-4865-8abc-a56d7e6c1554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:44.182204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a4c20652-666a-4865-8abc-a56d7e6c1554 None None] MFA Rules not processed for user `26725b170a2b429d91e91d4cafb6ec75`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:44.183537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a4c20652-666a-4865-8abc-a56d7e6c1554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:44.259733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1187/2388] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:54:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:44.273936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337c9d9-c9de-4784-8770-4d455cbfef1e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:44.275706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6337c9d9-c9de-4784-8770-4d455cbfef1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:44.347448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337c9d9-c9de-4784-8770-4d455cbfef1e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:44.351478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7cfd8-207d-47a4-8baa-c35c7a1fc84c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:44.352132 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7cfd8-207d-47a4-8baa-c35c7a1fc84c service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:44.352421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7cfd8-207d-47a4-8baa-c35c7a1fc84c service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:44.352846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7cfd8-207d-47a4-8baa-c35c7a1fc84c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:44.365702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:44.374499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7cfd8-207d-47a4-8baa-c35c7a1fc84c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:44.376074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7cfd8-207d-47a4-8baa-c35c7a1fc84c service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:44.388538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1202/2389] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:54:44.473069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c20652-666a-4865-8abc-a56d7e6c1554 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:44.483806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c20652-666a-4865-8abc-a56d7e6c1554 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:44.487515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4749-c9c9-4cf6-a58e-60407c3353b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:44.488165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4749-c9c9-4cf6-a58e-60407c3353b0 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:44.488449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4749-c9c9-4cf6-a58e-60407c3353b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:44.488774 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4749-c9c9-4cf6-a58e-60407c3353b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:44.523712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:54:44.554762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd4749-c9c9-4cf6-a58e-60407c3353b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:44.556263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd4749-c9c9-4cf6-a58e-60407c3353b0 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:44.566252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1188/2390] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:54:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:54:51.327853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784d15-6f25-4740-b77b-9b05c8a10de6 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:51.328354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784d15-6f25-4740-b77b-9b05c8a10de6 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:51.328454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce784d15-6f25-4740-b77b-9b05c8a10de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:51.347728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ce784d15-6f25-4740-b77b-9b05c8a10de6 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:54:51.349341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ce784d15-6f25-4740-b77b-9b05c8a10de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:51.405724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1203/2391] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:54:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:54:51.451210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4749-c9c9-4cf6-a58e-60407c3353b0 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:54:51.452646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3bcd4749-c9c9-4cf6-a58e-60407c3353b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:54:51.562210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4749-c9c9-4cf6-a58e-60407c3353b0 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:54:51.568146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa3287-e745-4920-90b9-f3e34eeb304b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:54:51.569000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa3287-e745-4920-90b9-f3e34eeb304b service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:54:51.569432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa3287-e745-4920-90b9-f3e34eeb304b service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:54:51.569711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa3287-e745-4920-90b9-f3e34eeb304b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:54:51.578892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa3287-e745-4920-90b9-f3e34eeb304b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:54:51.580484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa3287-e745-4920-90b9-f3e34eeb304b service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:54:51.591058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1189/2392] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:54:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:55:01.398293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=27 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:01.586329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=49 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:04.791422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd98346-e838-4df4-b4da-525f78dfe04b None None] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:04.791912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd98346-e838-4df4-b4da-525f78dfe04b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:04.792010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd98346-e838-4df4-b4da-525f78dfe04b None None] PATH_INFO: `/` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:04.793984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1204/2393] 10.222.0.47 () {54 vars in 921 bytes} [Fri Jun 24 23:55:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 23:55:04.799059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a99de7-c9f1-4dfc-b556-1a2a6c85b6a8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:04.799530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a99de7-c9f1-4dfc-b556-1a2a6c85b6a8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:04.799651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a99de7-c9f1-4dfc-b556-1a2a6c85b6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:04.819295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d5a99de7-c9f1-4dfc-b556-1a2a6c85b6a8 None None] MFA Rules not processed for user `feb9bec9f1894e179e24a14b81b3f23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:04.820494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d5a99de7-c9f1-4dfc-b556-1a2a6c85b6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:04.892925 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1190/2394] 10.222.0.47 () {58 vars in 1018 bytes} [Fri Jun 24 23:55:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:04.905207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd98346-e838-4df4-b4da-525f78dfe04b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:04.906939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ccd98346-e838-4df4-b4da-525f78dfe04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:04.993536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd98346-e838-4df4-b4da-525f78dfe04b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:04.998062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647bdce-5453-44c4-b940-f7f1dab29716 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71e5ef9dc32f494791ba4addb5151232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:04.998701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647bdce-5453-44c4-b940-f7f1dab29716 service glance] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:04.998986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647bdce-5453-44c4-b940-f7f1dab29716 service glance] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:04.999262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647bdce-5453-44c4-b940-f7f1dab29716 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:05.009164 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647bdce-5453-44c4-b940-f7f1dab29716 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:05.010593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647bdce-5453-44c4-b940-f7f1dab29716 service glance] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:05.022070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1205/2395] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:06.808471 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57288186-ef57-438c-b7bf-394559a60efd None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:06.809515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57288186-ef57-438c-b7bf-394559a60efd None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:06.809807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57288186-ef57-438c-b7bf-394559a60efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:06.828277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-57288186-ef57-438c-b7bf-394559a60efd None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:06.829506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-57288186-ef57-438c-b7bf-394559a60efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:06.836953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1191/2396] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:55:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:06.852717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647bdce-5453-44c4-b940-f7f1dab29716 service glance] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:06.864477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647bdce-5453-44c4-b940-f7f1dab29716 service glance] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:06.869540 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb59b4-b3b8-404a-8291-10b0b6368bd4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:06.870172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb59b4-b3b8-404a-8291-10b0b6368bd4 service swift] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:06.870460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb59b4-b3b8-404a-8291-10b0b6368bd4 service swift] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:06.870740 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb59b4-b3b8-404a-8291-10b0b6368bd4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:06.879311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb59b4-b3b8-404a-8291-10b0b6368bd4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:06.880922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb59b4-b3b8-404a-8291-10b0b6368bd4 service swift] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:06.888987 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1206/2397] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:55:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:55:16.822140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=34 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:16.885745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=41 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:24.375712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95d571-cf57-4370-9ed9-68da9d519354 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:24.376290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95d571-cf57-4370-9ed9-68da9d519354 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:24.376290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95d571-cf57-4370-9ed9-68da9d519354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:24.395265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-2f95d571-cf57-4370-9ed9-68da9d519354 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:24.396487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2f95d571-cf57-4370-9ed9-68da9d519354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:24.403881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1192/2398] 10.222.0.47 () {58 vars in 967 bytes} [Fri Jun 24 23:55:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:55:24.416048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb59b4-b3b8-404a-8291-10b0b6368bd4 service swift] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:24.427868 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb59b4-b3b8-404a-8291-10b0b6368bd4 service swift] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:24.433133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e57ec-680c-4b57-8d51-bea68b57da61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:24.434375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e57ec-680c-4b57-8d51-bea68b57da61 service swift] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:24.434668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e57ec-680c-4b57-8d51-bea68b57da61 service swift] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:24.435012 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e57ec-680c-4b57-8d51-bea68b57da61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:24.443617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e57ec-680c-4b57-8d51-bea68b57da61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:24.445268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e57ec-680c-4b57-8d51-bea68b57da61 service swift] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:24.454499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1207/2399] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:55:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:55:25.685728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95d571-cf57-4370-9ed9-68da9d519354 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:25.747920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95d571-cf57-4370-9ed9-68da9d519354 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:25.752536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:25.753383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:25.753668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:25.753947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] PATH_INFO: `/v3/users/28456b998d2d4f7f847f22ad92082362` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:25.755129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28456b998d2d4f7f847f22ad92082362)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:25.757089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:25.803356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28456b998d2d4f7f847f22ad92082362'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:25.814979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28456b998d2d4f7f847f22ad92082362'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:25.821705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28456b998d2d4f7f847f22ad92082362'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:25.822363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28456b998d2d4f7f847f22ad92082362'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:25.829839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28456b998d2d4f7f847f22ad92082362'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:25.836215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1193/2400] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:25 2022] DELETE /identity/v3/users/28456b998d2d4f7f847f22ad92082362 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:25.841297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11228011-d469-43a8-b198-f95639e2bf34 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:25.841761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11228011-d469-43a8-b198-f95639e2bf34 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:25.841883 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11228011-d469-43a8-b198-f95639e2bf34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:25.858879 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-11228011-d469-43a8-b198-f95639e2bf34 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:25.860074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-11228011-d469-43a8-b198-f95639e2bf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:25.917524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1208/2401] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:25.927304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:25.928905 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:25.988205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3a729-d4c4-469a-bcf3-a621263c5c97 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:25.991893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abadb48-0372-48c8-9750-565221903e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:25.992547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abadb48-0372-48c8-9750-565221903e3f service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:25.992861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abadb48-0372-48c8-9750-565221903e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:25.993140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abadb48-0372-48c8-9750-565221903e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:26.002885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abadb48-0372-48c8-9750-565221903e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:26.004254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abadb48-0372-48c8-9750-565221903e3f service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:26.014344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1194/2402] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:26.280610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11228011-d469-43a8-b198-f95639e2bf34 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:26.282220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-11228011-d469-43a8-b198-f95639e2bf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:26.295095 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11228011-d469-43a8-b198-f95639e2bf34 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:26.301178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:26.302039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:26.302317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:26.302593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] PATH_INFO: `/v3/projects/2d3ca8443d7645cbb459b8c7656ac324` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:26.303842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d3ca8443d7645cbb459b8c7656ac324)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:26.316203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:26.356707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d3ca8443d7645cbb459b8c7656ac324', 'request_id': 'req-04870506-e837-41d1-9a97-f6693a68c2c3'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.364564 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d3ca8443d7645cbb459b8c7656ac324', 'request_id': 'req-04870506-e837-41d1-9a97-f6693a68c2c3'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.371771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d3ca8443d7645cbb459b8c7656ac324', 'request_id': 'req-04870506-e837-41d1-9a97-f6693a68c2c3'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.382860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] The token cache is being invalidate because project 2d3ca8443d7645cbb459b8c7656ac324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:26.383148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.383816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.385375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1209/2403] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:26 2022] DELETE /identity/v3/projects/2d3ca8443d7645cbb459b8c7656ac324 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:26.389898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abadb48-0372-48c8-9750-565221903e3f service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:26.401946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abadb48-0372-48c8-9750-565221903e3f service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:26.414363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:26.461481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:26.462424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:26.462791 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:26.463137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] PATH_INFO: `/v3/users/26725b170a2b429d91e91d4cafb6ec75` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:26.464405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26725b170a2b429d91e91d4cafb6ec75)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:26.467140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:26.520610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26725b170a2b429d91e91d4cafb6ec75'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.534296 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26725b170a2b429d91e91d4cafb6ec75'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.543164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26725b170a2b429d91e91d4cafb6ec75'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.544244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26725b170a2b429d91e91d4cafb6ec75'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.552711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77abff83-af57-47b2-a236-d8be6a040059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26725b170a2b429d91e91d4cafb6ec75'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.558878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1195/2404] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:26 2022] DELETE /identity/v3/users/26725b170a2b429d91e91d4cafb6ec75 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:26.844553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:26.846114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:26.859628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04870506-e837-41d1-9a97-f6693a68c2c3 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:26.915873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:26.916773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:26.917056 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:26.917328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] PATH_INFO: `/v3/projects/39072db4d26940b58c766f47abd0dbe5` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:26.918612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39072db4d26940b58c766f47abd0dbe5)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:26.920498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:26.931363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:26.969409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39072db4d26940b58c766f47abd0dbe5', 'request_id': 'req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.977354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39072db4d26940b58c766f47abd0dbe5', 'request_id': 'req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.983748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39072db4d26940b58c766f47abd0dbe5', 'request_id': 'req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.994407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] The token cache is being invalidate because project 39072db4d26940b58c766f47abd0dbe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:26.994726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.995364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-76351e5b-feea-4d6f-8cd5-a4b9c8fc7588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:26.998271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1210/2405] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:26 2022] DELETE /identity/v3/projects/39072db4d26940b58c766f47abd0dbe5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:27.002642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d3fb-8b8d-4caa-834e-7641f1db9446 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.003135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d3fb-8b8d-4caa-834e-7641f1db9446 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.003250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d3fb-8b8d-4caa-834e-7641f1db9446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.021399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-90f2d3fb-8b8d-4caa-834e-7641f1db9446 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:27.022601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-90f2d3fb-8b8d-4caa-834e-7641f1db9446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:27.081922 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1196/2406] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:27.091320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0f02b-8735-4bb9-a6ac-c3b7bc042cf3 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.091615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0f02b-8735-4bb9-a6ac-c3b7bc042cf3 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.091900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0f02b-8735-4bb9-a6ac-c3b7bc042cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.110294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d0f0f02b-8735-4bb9-a6ac-c3b7bc042cf3 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:27.111491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d0f0f02b-8735-4bb9-a6ac-c3b7bc042cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:27.119389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1211/2407] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:27.123872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d3fb-8b8d-4caa-834e-7641f1db9446 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.133404 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d3fb-8b8d-4caa-834e-7641f1db9446 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.138663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc70b7-ee5d-4e51-b2e9-3a346c139a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.139313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc70b7-ee5d-4e51-b2e9-3a346c139a0f admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.139591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc70b7-ee5d-4e51-b2e9-3a346c139a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.139863 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc70b7-ee5d-4e51-b2e9-3a346c139a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.140596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc70b7-ee5d-4e51-b2e9-3a346c139a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:55:27.140917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc70b7-ee5d-4e51-b2e9-3a346c139a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.142497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc70b7-ee5d-4e51-b2e9-3a346c139a0f admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.154891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1197/2408] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:55:27.158930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0f02b-8735-4bb9-a6ac-c3b7bc042cf3 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.167988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0f02b-8735-4bb9-a6ac-c3b7bc042cf3 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.173359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c1c93-078e-463e-80ed-a9122cf777d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.174038 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c1c93-078e-463e-80ed-a9122cf777d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.174319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c1c93-078e-463e-80ed-a9122cf777d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.174594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c1c93-078e-463e-80ed-a9122cf777d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.175304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c1c93-078e-463e-80ed-a9122cf777d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.177255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c1c93-078e-463e-80ed-a9122cf777d4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.213675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1212/2409] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:55:27 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:27.218356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc70b7-ee5d-4e51-b2e9-3a346c139a0f admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.228127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc70b7-ee5d-4e51-b2e9-3a346c139a0f admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.246582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:27.287387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ede5f0-709d-4f5e-9160-332986eb103d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.288100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ede5f0-709d-4f5e-9160-332986eb103d admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.288389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ede5f0-709d-4f5e-9160-332986eb103d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.288830 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ede5f0-709d-4f5e-9160-332986eb103d admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.289529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ede5f0-709d-4f5e-9160-332986eb103d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.291457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ede5f0-709d-4f5e-9160-332986eb103d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.311252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1198/2410] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:55:27 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:27.315294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c1c93-078e-463e-80ed-a9122cf777d4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.326118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c1c93-078e-463e-80ed-a9122cf777d4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.330698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcc8e3-6467-480e-b767-243972a9d637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.331352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcc8e3-6467-480e-b767-243972a9d637 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.331638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcc8e3-6467-480e-b767-243972a9d637 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.331919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcc8e3-6467-480e-b767-243972a9d637 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.332735 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcc8e3-6467-480e-b767-243972a9d637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.334290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdcc8e3-6467-480e-b767-243972a9d637 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.344860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1213/2411] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:27.348542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ede5f0-709d-4f5e-9160-332986eb103d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.358767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ede5f0-709d-4f5e-9160-332986eb103d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.363436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc95ac5-772e-46a8-b813-75a5f5e5d9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.364088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95ac5-772e-46a8-b813-75a5f5e5d9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.364372 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95ac5-772e-46a8-b813-75a5f5e5d9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.364696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95ac5-772e-46a8-b813-75a5f5e5d9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.365420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95ac5-772e-46a8-b813-75a5f5e5d9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.366998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95ac5-772e-46a8-b813-75a5f5e5d9d7 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.376899 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1199/2412] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:27.380879 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcc8e3-6467-480e-b767-243972a9d637 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.392073 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcc8e3-6467-480e-b767-243972a9d637 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.397268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5a655-a9ec-4230-a0b2-4a7b18fef3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.398182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5a655-a9ec-4230-a0b2-4a7b18fef3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.398470 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5a655-a9ec-4230-a0b2-4a7b18fef3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.398749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5a655-a9ec-4230-a0b2-4a7b18fef3bd admin admin] PATH_INFO: `/v3/projects/709988cf4e43455a810d1c90c54c603c/users/a47ac7ced65e43d9a266981f039897ba/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.413509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5a655-a9ec-4230-a0b2-4a7b18fef3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=709988cf4e43455a810d1c90c54c603c, user_id=a47ac7ced65e43d9a266981f039897ba, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.416192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5a655-a9ec-4230-a0b2-4a7b18fef3bd admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.429374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1214/2413] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:27 2022] PUT /identity/v3/projects/709988cf4e43455a810d1c90c54c603c/users/a47ac7ced65e43d9a266981f039897ba/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:27.440509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841487e-6166-480d-bfd2-8a3b960bfbbb None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.440861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841487e-6166-480d-bfd2-8a3b960bfbbb None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.441146 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841487e-6166-480d-bfd2-8a3b960bfbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.468103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-e841487e-6166-480d-bfd2-8a3b960bfbbb None None] MFA Rules not processed for user `a47ac7ced65e43d9a266981f039897ba`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:27.469408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e841487e-6166-480d-bfd2-8a3b960bfbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:27.532961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1200/2414] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:27.537455 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5a655-a9ec-4230-a0b2-4a7b18fef3bd admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.598453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5a655-a9ec-4230-a0b2-4a7b18fef3bd admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.603190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61664577-1535-42ab-80ad-e999125c4e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.603847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61664577-1535-42ab-80ad-e999125c4e00 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.604126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61664577-1535-42ab-80ad-e999125c4e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.604401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61664577-1535-42ab-80ad-e999125c4e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.605146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61664577-1535-42ab-80ad-e999125c4e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.607639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61664577-1535-42ab-80ad-e999125c4e00 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.634893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1215/2415] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:55:27 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:27.636526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841487e-6166-480d-bfd2-8a3b960bfbbb None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.647446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841487e-6166-480d-bfd2-8a3b960bfbbb None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.701987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e4fa2-4d33-4808-80d8-aa1f9334bbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.702648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e4fa2-4d33-4808-80d8-aa1f9334bbac admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.702928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e4fa2-4d33-4808-80d8-aa1f9334bbac admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.703286 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e4fa2-4d33-4808-80d8-aa1f9334bbac admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.703891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e4fa2-4d33-4808-80d8-aa1f9334bbac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.705824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e4fa2-4d33-4808-80d8-aa1f9334bbac admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.727684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1201/2416] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:55:27 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:27.732303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61664577-1535-42ab-80ad-e999125c4e00 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.744823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61664577-1535-42ab-80ad-e999125c4e00 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.754212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c661446-c65d-4e90-acb4-15dfa0c87e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.754893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c661446-c65d-4e90-acb4-15dfa0c87e13 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.755179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c661446-c65d-4e90-acb4-15dfa0c87e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.755457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c661446-c65d-4e90-acb4-15dfa0c87e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.756197 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c661446-c65d-4e90-acb4-15dfa0c87e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.757854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c661446-c65d-4e90-acb4-15dfa0c87e13 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.769737 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1216/2417] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:27.773736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e4fa2-4d33-4808-80d8-aa1f9334bbac admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.784407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e4fa2-4d33-4808-80d8-aa1f9334bbac admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.789204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280ed3b-c0b4-4e96-884c-3dad64da4903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.790080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280ed3b-c0b4-4e96-884c-3dad64da4903 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.790363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280ed3b-c0b4-4e96-884c-3dad64da4903 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.790642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280ed3b-c0b4-4e96-884c-3dad64da4903 admin admin] PATH_INFO: `/v3/projects/342747e8df174af3a837e2487972ff09/users/6c183831cce74fe4a3e303712eb120e2/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.806906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280ed3b-c0b4-4e96-884c-3dad64da4903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=342747e8df174af3a837e2487972ff09, user_id=6c183831cce74fe4a3e303712eb120e2, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.810673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280ed3b-c0b4-4e96-884c-3dad64da4903 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.815219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:27.828648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1202/2418] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:27 2022] PUT /identity/v3/projects/342747e8df174af3a837e2487972ff09/users/6c183831cce74fe4a3e303712eb120e2/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:27.832717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c661446-c65d-4e90-acb4-15dfa0c87e13 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.838076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:27.845561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c661446-c65d-4e90-acb4-15dfa0c87e13 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.905240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd9905-9e46-4bfa-9574-00c2f76f0a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.905856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd9905-9e46-4bfa-9574-00c2f76f0a46 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.906140 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd9905-9e46-4bfa-9574-00c2f76f0a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.906396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd9905-9e46-4bfa-9574-00c2f76f0a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.907263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd9905-9e46-4bfa-9574-00c2f76f0a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.909142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd9905-9e46-4bfa-9574-00c2f76f0a46 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.919652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1217/2419] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:55:27.923781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280ed3b-c0b4-4e96-884c-3dad64da4903 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:27.935689 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280ed3b-c0b4-4e96-884c-3dad64da4903 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:27.940690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37f10-0a5e-41f3-b092-9fca8e0f9427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:27.941586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37f10-0a5e-41f3-b092-9fca8e0f9427 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.941867 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37f10-0a5e-41f3-b092-9fca8e0f9427 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.942143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37f10-0a5e-41f3-b092-9fca8e0f9427 admin admin] PATH_INFO: `/v3/projects/342747e8df174af3a837e2487972ff09/users/6c183831cce74fe4a3e303712eb120e2/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.944122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf37f10-0a5e-41f3-b092-9fca8e0f9427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=342747e8df174af3a837e2487972ff09, user_id=6c183831cce74fe4a3e303712eb120e2, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:27.946817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf37f10-0a5e-41f3-b092-9fca8e0f9427 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:27.958870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1203/2420] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:27 2022] PUT /identity/v3/projects/342747e8df174af3a837e2487972ff09/users/6c183831cce74fe4a3e303712eb120e2/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:55:27.969793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c396f4-dede-42b9-8157-9b9d22900358 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:27.970138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c396f4-dede-42b9-8157-9b9d22900358 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:27.970419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c396f4-dede-42b9-8157-9b9d22900358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:27.998684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-e2c396f4-dede-42b9-8157-9b9d22900358 None None] MFA Rules not processed for user `6c183831cce74fe4a3e303712eb120e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:27.999911 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e2c396f4-dede-42b9-8157-9b9d22900358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:28.087858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1218/2421] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:28.100923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37f10-0a5e-41f3-b092-9fca8e0f9427 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:28.102846 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-eaf37f10-0a5e-41f3-b092-9fca8e0f9427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:28.162766 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37f10-0a5e-41f3-b092-9fca8e0f9427 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:28.166432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6733-469f-4123-a10b-f5717059baa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:28.167074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6733-469f-4123-a10b-f5717059baa6 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:28.167503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6733-469f-4123-a10b-f5717059baa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:28.167600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6733-469f-4123-a10b-f5717059baa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:28.177463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f6733-469f-4123-a10b-f5717059baa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:28.178884 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f6733-469f-4123-a10b-f5717059baa6 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:28.188710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1204/2422] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:28.259001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c396f4-dede-42b9-8157-9b9d22900358 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:28.268924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c396f4-dede-42b9-8157-9b9d22900358 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:28.272455 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7795485-c771-4890-b851-458c9906040b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:28.273121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7795485-c771-4890-b851-458c9906040b service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:28.273401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7795485-c771-4890-b851-458c9906040b service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:28.273674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7795485-c771-4890-b851-458c9906040b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:28.329644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7795485-c771-4890-b851-458c9906040b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:28.331080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7795485-c771-4890-b851-458c9906040b service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:28.340357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1219/2423] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:31.037042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6733-469f-4123-a10b-f5717059baa6 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:31.100336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6733-469f-4123-a10b-f5717059baa6 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:31.104985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:31.105917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:31.106187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:31.106467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] PATH_INFO: `/v3/users/a47ac7ced65e43d9a266981f039897ba` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:31.107652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47ac7ced65e43d9a266981f039897ba)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:31.109802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:31.154657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47ac7ced65e43d9a266981f039897ba'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.166242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47ac7ced65e43d9a266981f039897ba'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.173604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47ac7ced65e43d9a266981f039897ba'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.174145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47ac7ced65e43d9a266981f039897ba'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.180983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47ac7ced65e43d9a266981f039897ba'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.187605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1205/2424] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:31 2022] DELETE /identity/v3/users/a47ac7ced65e43d9a266981f039897ba => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:31.192199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048ec38-d046-4406-a462-4c786a80ea2a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:31.192686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048ec38-d046-4406-a462-4c786a80ea2a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:31.192801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048ec38-d046-4406-a462-4c786a80ea2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:31.210034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3048ec38-d046-4406-a462-4c786a80ea2a None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:31.211297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3048ec38-d046-4406-a462-4c786a80ea2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:31.229087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:31.272839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1220/2425] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:31.283219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:31.284652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:31.291209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:31.345268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5f7a5-c83d-4f14-a943-a972def42cac admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:31.348858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a2d33-155a-4019-9d17-8f26442fd919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:31.349509 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a2d33-155a-4019-9d17-8f26442fd919 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:31.349803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a2d33-155a-4019-9d17-8f26442fd919 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:31.350056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a2d33-155a-4019-9d17-8f26442fd919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:31.359473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a2d33-155a-4019-9d17-8f26442fd919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:31.360926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a2d33-155a-4019-9d17-8f26442fd919 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:31.370143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1206/2426] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:31.646315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048ec38-d046-4406-a462-4c786a80ea2a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:31.647855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3048ec38-d046-4406-a462-4c786a80ea2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:31.663017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048ec38-d046-4406-a462-4c786a80ea2a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:31.668273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:31.669228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:31.669508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:31.669798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] PATH_INFO: `/v3/projects/709988cf4e43455a810d1c90c54c603c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:31.671042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=709988cf4e43455a810d1c90c54c603c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:31.672909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:31.719617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '709988cf4e43455a810d1c90c54c603c', 'request_id': 'req-05899612-a277-4bba-ba34-f14660881bd4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.726719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '709988cf4e43455a810d1c90c54c603c', 'request_id': 'req-05899612-a277-4bba-ba34-f14660881bd4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.733656 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '709988cf4e43455a810d1c90c54c603c', 'request_id': 'req-05899612-a277-4bba-ba34-f14660881bd4'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.744691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] The token cache is being invalidate because project 709988cf4e43455a810d1c90c54c603c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:31.744981 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.745619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.748321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1221/2427] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:31 2022] DELETE /identity/v3/projects/709988cf4e43455a810d1c90c54c603c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:31.752269 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a2d33-155a-4019-9d17-8f26442fd919 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:31.762220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a2d33-155a-4019-9d17-8f26442fd919 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:31.817507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:31.818370 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:31.818648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:31.818921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] PATH_INFO: `/v3/users/6c183831cce74fe4a3e303712eb120e2` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:31.820116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c183831cce74fe4a3e303712eb120e2)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:31.822045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:31.867200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c183831cce74fe4a3e303712eb120e2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.879035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c183831cce74fe4a3e303712eb120e2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.886256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c183831cce74fe4a3e303712eb120e2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.887489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c183831cce74fe4a3e303712eb120e2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.894522 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-83271e43-7e40-4737-b124-8a58652534f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c183831cce74fe4a3e303712eb120e2'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:31.900354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1207/2428] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:31 2022] DELETE /identity/v3/users/6c183831cce74fe4a3e303712eb120e2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:32.181145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:32.182588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:32.195553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05899612-a277-4bba-ba34-f14660881bd4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:32.255082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:32.256605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.257162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.257579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] PATH_INFO: `/v3/projects/342747e8df174af3a837e2487972ff09` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.258822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=342747e8df174af3a837e2487972ff09)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:32.261241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:32.309836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:32.311052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '342747e8df174af3a837e2487972ff09', 'request_id': 'req-3531f85c-1b26-4992-b92f-4487bbd0f2bb'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:32.318363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '342747e8df174af3a837e2487972ff09', 'request_id': 'req-3531f85c-1b26-4992-b92f-4487bbd0f2bb'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:32.325698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '342747e8df174af3a837e2487972ff09', 'request_id': 'req-3531f85c-1b26-4992-b92f-4487bbd0f2bb'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:32.335996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] The token cache is being invalidate because project 342747e8df174af3a837e2487972ff09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:32.336286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:32.336926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3531f85c-1b26-4992-b92f-4487bbd0f2bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:32.338471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1222/2429] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:32 2022] DELETE /identity/v3/projects/342747e8df174af3a837e2487972ff09 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:32.343987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cf942-fc78-4de5-850b-a696ccb35420 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.344445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cf942-fc78-4de5-850b-a696ccb35420 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.344539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cf942-fc78-4de5-850b-a696ccb35420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.361574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a64cf942-fc78-4de5-850b-a696ccb35420 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:32.362788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a64cf942-fc78-4de5-850b-a696ccb35420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:32.384032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:32.425216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1208/2430] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:32.432938 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9337e-50c1-4608-88df-014cef41df34 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.433228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9337e-50c1-4608-88df-014cef41df34 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.434007 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9337e-50c1-4608-88df-014cef41df34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.454099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3bb9337e-50c1-4608-88df-014cef41df34 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:32.455318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3bb9337e-50c1-4608-88df-014cef41df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:32.463095 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1223/2431] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:32.467853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cf942-fc78-4de5-850b-a696ccb35420 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:32.477866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cf942-fc78-4de5-850b-a696ccb35420 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:32.482568 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9e16-3672-4da8-944f-962c4a1db9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:32.483254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9e16-3672-4da8-944f-962c4a1db9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.483539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9e16-3672-4da8-944f-962c4a1db9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.483820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9e16-3672-4da8-944f-962c4a1db9e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.484606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9e16-3672-4da8-944f-962c4a1db9e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:55:32.484928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9e16-3672-4da8-944f-962c4a1db9e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:32.486563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9e16-3672-4da8-944f-962c4a1db9e6 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:32.498217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1209/2432] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:55:32.502330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9337e-50c1-4608-88df-014cef41df34 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:32.512626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9337e-50c1-4608-88df-014cef41df34 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:32.517396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8183a-f61e-4277-94be-1df90d42b12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:32.518043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8183a-f61e-4277-94be-1df90d42b12f admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.518586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8183a-f61e-4277-94be-1df90d42b12f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.518861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8183a-f61e-4277-94be-1df90d42b12f admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.519580 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8183a-f61e-4277-94be-1df90d42b12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:32.521583 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8183a-f61e-4277-94be-1df90d42b12f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:32.548570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1224/2433] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:55:32 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:32.552736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9e16-3672-4da8-944f-962c4a1db9e6 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:32.564197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9e16-3672-4da8-944f-962c4a1db9e6 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:32.621583 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcdf77-64fc-45e5-ac50-be2cb8a3098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:32.622252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcdf77-64fc-45e5-ac50-be2cb8a3098a admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.622540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcdf77-64fc-45e5-ac50-be2cb8a3098a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.622827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcdf77-64fc-45e5-ac50-be2cb8a3098a admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.623515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcdf77-64fc-45e5-ac50-be2cb8a3098a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:32.625497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcdf77-64fc-45e5-ac50-be2cb8a3098a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:32.645160 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1210/2434] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:55:32 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:32.649534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8183a-f61e-4277-94be-1df90d42b12f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:32.659871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8183a-f61e-4277-94be-1df90d42b12f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:32.664466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee9c74-3eb8-4de6-920a-cc43ada25b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:32.665148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee9c74-3eb8-4de6-920a-cc43ada25b1e admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.665429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee9c74-3eb8-4de6-920a-cc43ada25b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.665705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee9c74-3eb8-4de6-920a-cc43ada25b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.666454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee9c74-3eb8-4de6-920a-cc43ada25b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:32.668525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee9c74-3eb8-4de6-920a-cc43ada25b1e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:32.678905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1225/2435] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:32.682651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcdf77-64fc-45e5-ac50-be2cb8a3098a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:32.695826 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcdf77-64fc-45e5-ac50-be2cb8a3098a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:32.701501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf81dd2-c106-49f6-933e-997b6e075fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:32.702158 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf81dd2-c106-49f6-933e-997b6e075fef admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.702438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf81dd2-c106-49f6-933e-997b6e075fef admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.702712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf81dd2-c106-49f6-933e-997b6e075fef admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.703442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf81dd2-c106-49f6-933e-997b6e075fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:32.705102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf81dd2-c106-49f6-933e-997b6e075fef admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:32.715719 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1211/2436] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:32.719818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee9c74-3eb8-4de6-920a-cc43ada25b1e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:32.729818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee9c74-3eb8-4de6-920a-cc43ada25b1e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:32.735061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e5f5b-5af5-4178-8886-b7f77a91adc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:32.735924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e5f5b-5af5-4178-8886-b7f77a91adc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.736204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e5f5b-5af5-4178-8886-b7f77a91adc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.736477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e5f5b-5af5-4178-8886-b7f77a91adc6 admin admin] PATH_INFO: `/v3/projects/ed6f4077c7a54caa868865c8cf100315/users/082ec66381a84eedbc77ce9ce99215a8/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.752816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e5f5b-5af5-4178-8886-b7f77a91adc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed6f4077c7a54caa868865c8cf100315, user_id=082ec66381a84eedbc77ce9ce99215a8, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:32.755386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5e5f5b-5af5-4178-8886-b7f77a91adc6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:32.767920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1226/2437] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:32 2022] PUT /identity/v3/projects/ed6f4077c7a54caa868865c8cf100315/users/082ec66381a84eedbc77ce9ce99215a8/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:55:32.778620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d70601-d920-4002-bd7c-2a6229303930 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.778916 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d70601-d920-4002-bd7c-2a6229303930 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.779205 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d70601-d920-4002-bd7c-2a6229303930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.805975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-93d70601-d920-4002-bd7c-2a6229303930 None None] MFA Rules not processed for user `082ec66381a84eedbc77ce9ce99215a8`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:32.807225 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-93d70601-d920-4002-bd7c-2a6229303930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:32.875374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1212/2438] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:32.880995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e5f5b-5af5-4178-8886-b7f77a91adc6 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:32.948561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e5f5b-5af5-4178-8886-b7f77a91adc6 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:32.954016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28ce97-73d6-40fc-8333-ae8dfd17e006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:32.954674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28ce97-73d6-40fc-8333-ae8dfd17e006 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:32.954988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28ce97-73d6-40fc-8333-ae8dfd17e006 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:32.955276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28ce97-73d6-40fc-8333-ae8dfd17e006 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:32.955988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28ce97-73d6-40fc-8333-ae8dfd17e006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:32.957930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28ce97-73d6-40fc-8333-ae8dfd17e006 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:32.987867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1227/2439] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:55:32 2022] POST /identity/v3/projects => generated 397 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:32.989089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d70601-d920-4002-bd7c-2a6229303930 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:32.999865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d70601-d920-4002-bd7c-2a6229303930 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:33.019080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:33.063956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52861ee0-d415-44ed-91d1-780ca8c6a610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:33.064668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52861ee0-d415-44ed-91d1-780ca8c6a610 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:33.064946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52861ee0-d415-44ed-91d1-780ca8c6a610 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:33.065226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52861ee0-d415-44ed-91d1-780ca8c6a610 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:33.065919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52861ee0-d415-44ed-91d1-780ca8c6a610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:33.067839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52861ee0-d415-44ed-91d1-780ca8c6a610 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:33.091937 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1213/2440] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:55:32 2022] POST /identity/v3/users => generated 305 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:33.097045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28ce97-73d6-40fc-8333-ae8dfd17e006 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:33.109338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28ce97-73d6-40fc-8333-ae8dfd17e006 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:33.114137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f72be-3cd1-4b97-b29f-929886093e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:33.114796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f72be-3cd1-4b97-b29f-929886093e9b admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:33.115077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f72be-3cd1-4b97-b29f-929886093e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:33.115390 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f72be-3cd1-4b97-b29f-929886093e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:33.116143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f72be-3cd1-4b97-b29f-929886093e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:33.117906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f72be-3cd1-4b97-b29f-929886093e9b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:33.129688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1228/2441] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:33.134115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52861ee0-d415-44ed-91d1-780ca8c6a610 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:33.146016 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52861ee0-d415-44ed-91d1-780ca8c6a610 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:33.151262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9ddaa-bfe2-490e-9463-17f86c1e63bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:33.152325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9ddaa-bfe2-490e-9463-17f86c1e63bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:33.152771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9ddaa-bfe2-490e-9463-17f86c1e63bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:33.153054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9ddaa-bfe2-490e-9463-17f86c1e63bc admin admin] PATH_INFO: `/v3/projects/db1feb76e54145febfdeb60728ce0fa1/users/a0f70c8b4a994f15a0625c05af6995f5/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:33.170732 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9ddaa-bfe2-490e-9463-17f86c1e63bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1feb76e54145febfdeb60728ce0fa1, user_id=a0f70c8b4a994f15a0625c05af6995f5, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:33.173754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9ddaa-bfe2-490e-9463-17f86c1e63bc admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:33.197482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f72be-3cd1-4b97-b29f-929886093e9b admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:33.197969 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1214/2442] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:33 2022] PUT /identity/v3/projects/db1feb76e54145febfdeb60728ce0fa1/users/a0f70c8b4a994f15a0625c05af6995f5/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:33.211912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f72be-3cd1-4b97-b29f-929886093e9b admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:33.259931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:33.275406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f54d6-8704-4785-8f3d-2d5250167a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:33.276001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f54d6-8704-4785-8f3d-2d5250167a95 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:33.276282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f54d6-8704-4785-8f3d-2d5250167a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:33.276556 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f54d6-8704-4785-8f3d-2d5250167a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:33.277961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f54d6-8704-4785-8f3d-2d5250167a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:33.279553 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f54d6-8704-4785-8f3d-2d5250167a95 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:33.289194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1229/2443] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:33.293616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9ddaa-bfe2-490e-9463-17f86c1e63bc admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:33.304268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9ddaa-bfe2-490e-9463-17f86c1e63bc admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:33.314555 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c5b39-3efa-4e96-ba9c-6af09d2c0c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:33.315463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c5b39-3efa-4e96-ba9c-6af09d2c0c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:33.315754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c5b39-3efa-4e96-ba9c-6af09d2c0c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:33.316034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c5b39-3efa-4e96-ba9c-6af09d2c0c7e admin admin] PATH_INFO: `/v3/projects/db1feb76e54145febfdeb60728ce0fa1/users/a0f70c8b4a994f15a0625c05af6995f5/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:33.318086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c5b39-3efa-4e96-ba9c-6af09d2c0c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1feb76e54145febfdeb60728ce0fa1, user_id=a0f70c8b4a994f15a0625c05af6995f5, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:33.320801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c5b39-3efa-4e96-ba9c-6af09d2c0c7e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:33.332295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1215/2444] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:33 2022] PUT /identity/v3/projects/db1feb76e54145febfdeb60728ce0fa1/users/a0f70c8b4a994f15a0625c05af6995f5/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:33.344382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50113857-c8bc-4b2f-839d-c672514a8156 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:33.344696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50113857-c8bc-4b2f-839d-c672514a8156 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:33.344979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50113857-c8bc-4b2f-839d-c672514a8156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:33.371132 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-50113857-c8bc-4b2f-839d-c672514a8156 None None] MFA Rules not processed for user `a0f70c8b4a994f15a0625c05af6995f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:33.372343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-50113857-c8bc-4b2f-839d-c672514a8156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:33.440721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1230/2445] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:33.451992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c5b39-3efa-4e96-ba9c-6af09d2c0c7e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:33.454354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-136c5b39-3efa-4e96-ba9c-6af09d2c0c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:33.511877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c5b39-3efa-4e96-ba9c-6af09d2c0c7e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:33.515612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09da4d-2f17-4f7d-8dab-6bd763456589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:33.516304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09da4d-2f17-4f7d-8dab-6bd763456589 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:33.516611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09da4d-2f17-4f7d-8dab-6bd763456589 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:33.516891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09da4d-2f17-4f7d-8dab-6bd763456589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:33.526058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09da4d-2f17-4f7d-8dab-6bd763456589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:33.528018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09da4d-2f17-4f7d-8dab-6bd763456589 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:33.537484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1216/2446] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:33.608778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50113857-c8bc-4b2f-839d-c672514a8156 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:33.618417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50113857-c8bc-4b2f-839d-c672514a8156 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:33.622138 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ead3e-511e-4a25-bd48-fbefc7ab125e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:33.622771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ead3e-511e-4a25-bd48-fbefc7ab125e service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:33.623051 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ead3e-511e-4a25-bd48-fbefc7ab125e service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:33.623325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ead3e-511e-4a25-bd48-fbefc7ab125e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:33.678113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ead3e-511e-4a25-bd48-fbefc7ab125e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:33.679550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ead3e-511e-4a25-bd48-fbefc7ab125e service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:33.689058 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1231/2447] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:37.701913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09da4d-2f17-4f7d-8dab-6bd763456589 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:37.764790 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09da4d-2f17-4f7d-8dab-6bd763456589 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:37.769679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:37.770547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:37.770831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:37.771107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] PATH_INFO: `/v3/users/082ec66381a84eedbc77ce9ce99215a8` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:37.772293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=082ec66381a84eedbc77ce9ce99215a8)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:37.774245 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:37.795219 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:37.820739 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '082ec66381a84eedbc77ce9ce99215a8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:37.832510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '082ec66381a84eedbc77ce9ce99215a8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:37.839557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '082ec66381a84eedbc77ce9ce99215a8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:37.840340 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '082ec66381a84eedbc77ce9ce99215a8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:37.847275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '082ec66381a84eedbc77ce9ce99215a8'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:37.853080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1217/2448] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:37 2022] DELETE /identity/v3/users/082ec66381a84eedbc77ce9ce99215a8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:37.858192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c686a-5930-4afe-bc9b-41346123538e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:37.858651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c686a-5930-4afe-bc9b-41346123538e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:37.858745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c686a-5930-4afe-bc9b-41346123538e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:37.876137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-280c686a-5930-4afe-bc9b-41346123538e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:37.877376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-280c686a-5930-4afe-bc9b-41346123538e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:37.941995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1232/2449] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:37.952044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:37.953527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:38.011317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894077a5-acef-40ab-9094-7d1312368cf8 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:38.014981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598212df-0f55-4c0f-88f1-67d7fc0be34c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:38.015617 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598212df-0f55-4c0f-88f1-67d7fc0be34c service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:38.015894 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598212df-0f55-4c0f-88f1-67d7fc0be34c service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:38.016167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598212df-0f55-4c0f-88f1-67d7fc0be34c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:38.025336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598212df-0f55-4c0f-88f1-67d7fc0be34c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:38.027265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598212df-0f55-4c0f-88f1-67d7fc0be34c service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:38.036523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1218/2450] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:55:38.309926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c686a-5930-4afe-bc9b-41346123538e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:38.311503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-280c686a-5930-4afe-bc9b-41346123538e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:38.326183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c686a-5930-4afe-bc9b-41346123538e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:38.331241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:38.332146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:38.332440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:38.332768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] PATH_INFO: `/v3/projects/ed6f4077c7a54caa868865c8cf100315` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:38.333977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed6f4077c7a54caa868865c8cf100315)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:38.335894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:38.338172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:38.388632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed6f4077c7a54caa868865c8cf100315', 'request_id': 'req-97023a80-a6c5-453e-af18-040aa750f29f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.396020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed6f4077c7a54caa868865c8cf100315', 'request_id': 'req-97023a80-a6c5-453e-af18-040aa750f29f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.403336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed6f4077c7a54caa868865c8cf100315', 'request_id': 'req-97023a80-a6c5-453e-af18-040aa750f29f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.413623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] The token cache is being invalidate because project ed6f4077c7a54caa868865c8cf100315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:38.413912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.414546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.416078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1233/2451] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:38 2022] DELETE /identity/v3/projects/ed6f4077c7a54caa868865c8cf100315 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:38.420524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598212df-0f55-4c0f-88f1-67d7fc0be34c service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:38.430230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598212df-0f55-4c0f-88f1-67d7fc0be34c service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:38.488105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873872c-7712-4c37-8941-494728295965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:38.488997 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873872c-7712-4c37-8941-494728295965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:38.489276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873872c-7712-4c37-8941-494728295965 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:38.489550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873872c-7712-4c37-8941-494728295965 admin admin] PATH_INFO: `/v3/users/a0f70c8b4a994f15a0625c05af6995f5` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:38.490712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873872c-7712-4c37-8941-494728295965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f70c8b4a994f15a0625c05af6995f5)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:38.492670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873872c-7712-4c37-8941-494728295965 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:38.535607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:38.541588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d873872c-7712-4c37-8941-494728295965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f70c8b4a994f15a0625c05af6995f5'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.554042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d873872c-7712-4c37-8941-494728295965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f70c8b4a994f15a0625c05af6995f5'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.561605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d873872c-7712-4c37-8941-494728295965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f70c8b4a994f15a0625c05af6995f5'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.562224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d873872c-7712-4c37-8941-494728295965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f70c8b4a994f15a0625c05af6995f5'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.569211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d873872c-7712-4c37-8941-494728295965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f70c8b4a994f15a0625c05af6995f5'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:38.576127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1219/2452] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:38 2022] DELETE /identity/v3/users/a0f70c8b4a994f15a0625c05af6995f5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:38.873509 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:38.875066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:38.888705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97023a80-a6c5-453e-af18-040aa750f29f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:38.954920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:38.956014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:38.956304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:38.956599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] PATH_INFO: `/v3/projects/db1feb76e54145febfdeb60728ce0fa1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:38.957945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1feb76e54145febfdeb60728ce0fa1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:38.959915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.014366 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1feb76e54145febfdeb60728ce0fa1', 'request_id': 'req-957096ab-3874-4d5a-9455-735a894769cd'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:39.021973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1feb76e54145febfdeb60728ce0fa1', 'request_id': 'req-957096ab-3874-4d5a-9455-735a894769cd'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:39.028797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1feb76e54145febfdeb60728ce0fa1', 'request_id': 'req-957096ab-3874-4d5a-9455-735a894769cd'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:39.038991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] The token cache is being invalidate because project db1feb76e54145febfdeb60728ce0fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:39.039274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:39.039908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-957096ab-3874-4d5a-9455-735a894769cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:39.041460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1234/2453] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:38 2022] DELETE /identity/v3/projects/db1feb76e54145febfdeb60728ce0fa1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:39.046906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c1bb6-b188-4432-86d7-dd4c0879ab2e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.047396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c1bb6-b188-4432-86d7-dd4c0879ab2e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.047486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c1bb6-b188-4432-86d7-dd4c0879ab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.065111 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-b19c1bb6-b188-4432-86d7-dd4c0879ab2e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:39.066288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b19c1bb6-b188-4432-86d7-dd4c0879ab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:39.126028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1220/2454] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:39.139798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318036b-1902-49c9-aaa0-0a8dfd27e3c9 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.140088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318036b-1902-49c9-aaa0-0a8dfd27e3c9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.140366 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318036b-1902-49c9-aaa0-0a8dfd27e3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.159701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2318036b-1902-49c9-aaa0-0a8dfd27e3c9 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:39.160983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2318036b-1902-49c9-aaa0-0a8dfd27e3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:39.168967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1235/2455] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:55:39.173206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c1bb6-b188-4432-86d7-dd4c0879ab2e None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.182890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c1bb6-b188-4432-86d7-dd4c0879ab2e None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.187635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5183f4-1da0-4cc1-9f14-c46e502bc74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.188289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5183f4-1da0-4cc1-9f14-c46e502bc74a admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.188603 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5183f4-1da0-4cc1-9f14-c46e502bc74a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.188892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5183f4-1da0-4cc1-9f14-c46e502bc74a admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.189648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5183f4-1da0-4cc1-9f14-c46e502bc74a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:55:39.189961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5183f4-1da0-4cc1-9f14-c46e502bc74a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.191584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5183f4-1da0-4cc1-9f14-c46e502bc74a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.203863 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1221/2456] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:55:39.208209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318036b-1902-49c9-aaa0-0a8dfd27e3c9 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.218190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318036b-1902-49c9-aaa0-0a8dfd27e3c9 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.222905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5a6bb-58e7-4fc0-b0ec-de07539c72f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.223592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5a6bb-58e7-4fc0-b0ec-de07539c72f6 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.223872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5a6bb-58e7-4fc0-b0ec-de07539c72f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.224146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5a6bb-58e7-4fc0-b0ec-de07539c72f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.224873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5a6bb-58e7-4fc0-b0ec-de07539c72f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.226773 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5a6bb-58e7-4fc0-b0ec-de07539c72f6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.236461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:39.253673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1236/2457] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:55:39 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:39.257883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5183f4-1da0-4cc1-9f14-c46e502bc74a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.269004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5183f4-1da0-4cc1-9f14-c46e502bc74a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.325165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d4af3-da36-45ad-b012-41443ddcca5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.325829 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d4af3-da36-45ad-b012-41443ddcca5e admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.326112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d4af3-da36-45ad-b012-41443ddcca5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.326395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d4af3-da36-45ad-b012-41443ddcca5e admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.327077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d4af3-da36-45ad-b012-41443ddcca5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.329014 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d4af3-da36-45ad-b012-41443ddcca5e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.350795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1222/2458] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:55:39 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:39.352217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5a6bb-58e7-4fc0-b0ec-de07539c72f6 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.363203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5a6bb-58e7-4fc0-b0ec-de07539c72f6 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.368034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d5e7e-8a8c-494e-ae7b-51552bc175bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.368724 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d5e7e-8a8c-494e-ae7b-51552bc175bd admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.369013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d5e7e-8a8c-494e-ae7b-51552bc175bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.369324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d5e7e-8a8c-494e-ae7b-51552bc175bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.370049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d5e7e-8a8c-494e-ae7b-51552bc175bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.372362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d5e7e-8a8c-494e-ae7b-51552bc175bd admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.382645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1237/2459] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:39.386507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d4af3-da36-45ad-b012-41443ddcca5e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.396697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d4af3-da36-45ad-b012-41443ddcca5e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.401364 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c41095-76f4-4dd4-abf8-d638111ba2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.402046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c41095-76f4-4dd4-abf8-d638111ba2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.402501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c41095-76f4-4dd4-abf8-d638111ba2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.402786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c41095-76f4-4dd4-abf8-d638111ba2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.403514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c41095-76f4-4dd4-abf8-d638111ba2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.405129 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c41095-76f4-4dd4-abf8-d638111ba2d0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.415075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1223/2460] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:39.419411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d5e7e-8a8c-494e-ae7b-51552bc175bd admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.429386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d5e7e-8a8c-494e-ae7b-51552bc175bd admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.435079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2454a52-6142-44de-9854-ddd806cf92bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.435986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2454a52-6142-44de-9854-ddd806cf92bf admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.436272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2454a52-6142-44de-9854-ddd806cf92bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.436548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2454a52-6142-44de-9854-ddd806cf92bf admin admin] PATH_INFO: `/v3/projects/6e2153c9eea648c890218b6c2b53257a/users/94ec055bf71641c596b67a4096fe43c9/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.451121 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2454a52-6142-44de-9854-ddd806cf92bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2153c9eea648c890218b6c2b53257a, user_id=94ec055bf71641c596b67a4096fe43c9, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.453805 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2454a52-6142-44de-9854-ddd806cf92bf admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.467752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1238/2461] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:39 2022] PUT /identity/v3/projects/6e2153c9eea648c890218b6c2b53257a/users/94ec055bf71641c596b67a4096fe43c9/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:39.478036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16bc36-ae6a-4c63-82f4-f1714c36b70f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.478341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16bc36-ae6a-4c63-82f4-f1714c36b70f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.478665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16bc36-ae6a-4c63-82f4-f1714c36b70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.505733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-0f16bc36-ae6a-4c63-82f4-f1714c36b70f None None] MFA Rules not processed for user `94ec055bf71641c596b67a4096fe43c9`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:39.506933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0f16bc36-ae6a-4c63-82f4-f1714c36b70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:39.519243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:39.573924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1224/2462] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:55:39.578311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2454a52-6142-44de-9854-ddd806cf92bf admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.638875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2454a52-6142-44de-9854-ddd806cf92bf admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.643670 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96623604-d61b-4c6d-852e-ec12622f1060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.644313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96623604-d61b-4c6d-852e-ec12622f1060 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.644606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96623604-d61b-4c6d-852e-ec12622f1060 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.645003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96623604-d61b-4c6d-852e-ec12622f1060 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.645681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96623604-d61b-4c6d-852e-ec12622f1060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.647578 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96623604-d61b-4c6d-852e-ec12622f1060 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.673683 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1239/2463] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:55:39 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:39.677520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16bc36-ae6a-4c63-82f4-f1714c36b70f None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.688171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16bc36-ae6a-4c63-82f4-f1714c36b70f None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.744427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31046de-418f-43d6-a9ef-5c6ed4025664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.745150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31046de-418f-43d6-a9ef-5c6ed4025664 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.745526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31046de-418f-43d6-a9ef-5c6ed4025664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.745915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31046de-418f-43d6-a9ef-5c6ed4025664 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.746613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31046de-418f-43d6-a9ef-5c6ed4025664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.748539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31046de-418f-43d6-a9ef-5c6ed4025664 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.769010 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1225/2464] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:55:39 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:39.772941 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96623604-d61b-4c6d-852e-ec12622f1060 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.783786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96623604-d61b-4c6d-852e-ec12622f1060 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.788413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b8af7-8c49-4b1d-a2e2-049e8a0c5288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.789119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b8af7-8c49-4b1d-a2e2-049e8a0c5288 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.789408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b8af7-8c49-4b1d-a2e2-049e8a0c5288 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.789688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b8af7-8c49-4b1d-a2e2-049e8a0c5288 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.790412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b8af7-8c49-4b1d-a2e2-049e8a0c5288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.792018 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b8af7-8c49-4b1d-a2e2-049e8a0c5288 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.802587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1240/2465] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:39.806684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31046de-418f-43d6-a9ef-5c6ed4025664 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.817171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31046de-418f-43d6-a9ef-5c6ed4025664 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.821821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371a383-ebcc-44c4-aa65-6ebc0cec418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.822694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371a383-ebcc-44c4-aa65-6ebc0cec418e admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.822983 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371a383-ebcc-44c4-aa65-6ebc0cec418e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.823264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371a383-ebcc-44c4-aa65-6ebc0cec418e admin admin] PATH_INFO: `/v3/projects/d21d15563f4a4f33bd319c90871d8573/users/051a2064ee494218a88b69e9a86c0196/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.838628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371a383-ebcc-44c4-aa65-6ebc0cec418e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21d15563f4a4f33bd319c90871d8573, user_id=051a2064ee494218a88b69e9a86c0196, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.841302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371a383-ebcc-44c4-aa65-6ebc0cec418e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.853960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1226/2466] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:39 2022] PUT /identity/v3/projects/d21d15563f4a4f33bd319c90871d8573/users/051a2064ee494218a88b69e9a86c0196/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:55:39.858323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b8af7-8c49-4b1d-a2e2-049e8a0c5288 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.868029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b8af7-8c49-4b1d-a2e2-049e8a0c5288 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.924727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5c004-e54b-45c2-999c-9944451398e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.925381 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5c004-e54b-45c2-999c-9944451398e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.925711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5c004-e54b-45c2-999c-9944451398e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.925984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5c004-e54b-45c2-999c-9944451398e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.926714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5c004-e54b-45c2-999c-9944451398e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.928315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5c004-e54b-45c2-999c-9944451398e6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.938999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1241/2467] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:39.943104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371a383-ebcc-44c4-aa65-6ebc0cec418e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:39.954400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371a383-ebcc-44c4-aa65-6ebc0cec418e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:39.959273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae0e0c-c27f-4a79-a667-c5a0c543729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:39.960167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae0e0c-c27f-4a79-a667-c5a0c543729e admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.960452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae0e0c-c27f-4a79-a667-c5a0c543729e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.960778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae0e0c-c27f-4a79-a667-c5a0c543729e admin admin] PATH_INFO: `/v3/projects/d21d15563f4a4f33bd319c90871d8573/users/051a2064ee494218a88b69e9a86c0196/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:39.962849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae0e0c-c27f-4a79-a667-c5a0c543729e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21d15563f4a4f33bd319c90871d8573, user_id=051a2064ee494218a88b69e9a86c0196, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:39.966831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae0e0c-c27f-4a79-a667-c5a0c543729e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:39.979440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1227/2468] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:39 2022] PUT /identity/v3/projects/d21d15563f4a4f33bd319c90871d8573/users/051a2064ee494218a88b69e9a86c0196/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:39.991675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cfa2d-8245-4b7f-8ce3-0b7eb93536ec None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:39.992082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cfa2d-8245-4b7f-8ce3-0b7eb93536ec None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:39.992374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cfa2d-8245-4b7f-8ce3-0b7eb93536ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:40.020363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b59cfa2d-8245-4b7f-8ce3-0b7eb93536ec None None] MFA Rules not processed for user `051a2064ee494218a88b69e9a86c0196`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:40.021623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b59cfa2d-8245-4b7f-8ce3-0b7eb93536ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:40.035373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:40.095862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1242/2469] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:40.107598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae0e0c-c27f-4a79-a667-c5a0c543729e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:40.109426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-afae0e0c-c27f-4a79-a667-c5a0c543729e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:40.174686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae0e0c-c27f-4a79-a667-c5a0c543729e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:40.178376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd8976-10e0-4358-b794-a42cf2fa3a6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:40.178956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd8976-10e0-4358-b794-a42cf2fa3a6d service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:40.179234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd8976-10e0-4358-b794-a42cf2fa3a6d service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:40.179508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd8976-10e0-4358-b794-a42cf2fa3a6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:40.189295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd8976-10e0-4358-b794-a42cf2fa3a6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:40.190706 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd8976-10e0-4358-b794-a42cf2fa3a6d service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:40.201007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1228/2470] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:40.287595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cfa2d-8245-4b7f-8ce3-0b7eb93536ec None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:40.298839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cfa2d-8245-4b7f-8ce3-0b7eb93536ec None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:40.304014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286dc5a-d44f-42f5-899a-d5a281ce2377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:40.305039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286dc5a-d44f-42f5-899a-d5a281ce2377 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:40.305453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286dc5a-d44f-42f5-899a-d5a281ce2377 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:40.305843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286dc5a-d44f-42f5-899a-d5a281ce2377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:40.382089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286dc5a-d44f-42f5-899a-d5a281ce2377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:40.383557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286dc5a-d44f-42f5-899a-d5a281ce2377 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:40.393000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1243/2471] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:45.169135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd8976-10e0-4358-b794-a42cf2fa3a6d service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:45.179670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:45.248805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd8976-10e0-4358-b794-a42cf2fa3a6d service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:45.253315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:45.254188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:45.254472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:45.254751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] PATH_INFO: `/v3/users/94ec055bf71641c596b67a4096fe43c9` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:45.255931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94ec055bf71641c596b67a4096fe43c9)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:45.257891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:45.303338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94ec055bf71641c596b67a4096fe43c9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:45.315209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94ec055bf71641c596b67a4096fe43c9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:45.322176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94ec055bf71641c596b67a4096fe43c9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:45.322859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94ec055bf71641c596b67a4096fe43c9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:45.330218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94ec055bf71641c596b67a4096fe43c9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:45.335995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1229/2472] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:45 2022] DELETE /identity/v3/users/94ec055bf71641c596b67a4096fe43c9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:45.341623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba62c5-44d9-4340-96a6-9c855d218025 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:45.342089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba62c5-44d9-4340-96a6-9c855d218025 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:45.342208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba62c5-44d9-4340-96a6-9c855d218025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:45.359431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-75ba62c5-44d9-4340-96a6-9c855d218025 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:45.360660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-75ba62c5-44d9-4340-96a6-9c855d218025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:45.420967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1244/2473] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:45.430546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:45.432088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:45.491288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f147-9080-4f33-b2c2-8b82b20b60f9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:45.494934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d46ada-28db-4625-aba2-805b43fc7733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:45.495609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d46ada-28db-4625-aba2-805b43fc7733 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:45.495895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d46ada-28db-4625-aba2-805b43fc7733 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:45.496169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d46ada-28db-4625-aba2-805b43fc7733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:45.505312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d46ada-28db-4625-aba2-805b43fc7733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:45.506664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d46ada-28db-4625-aba2-805b43fc7733 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:45.516067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1230/2474] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:45.956537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba62c5-44d9-4340-96a6-9c855d218025 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:45.958507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-75ba62c5-44d9-4340-96a6-9c855d218025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:45.972765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba62c5-44d9-4340-96a6-9c855d218025 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:45.977897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:45.978871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:45.979445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:45.980513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] PATH_INFO: `/v3/projects/6e2153c9eea648c890218b6c2b53257a` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:45.981823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2153c9eea648c890218b6c2b53257a)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:45.983857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:46.048828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:46.054142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2153c9eea648c890218b6c2b53257a', 'request_id': 'req-4cb2f24d-bced-4c42-b928-89e9fc60f47b'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.064505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2153c9eea648c890218b6c2b53257a', 'request_id': 'req-4cb2f24d-bced-4c42-b928-89e9fc60f47b'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.071878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2153c9eea648c890218b6c2b53257a', 'request_id': 'req-4cb2f24d-bced-4c42-b928-89e9fc60f47b'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.094733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] The token cache is being invalidate because project 6e2153c9eea648c890218b6c2b53257a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:46.095042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.095713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.097946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1245/2475] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:45 2022] DELETE /identity/v3/projects/6e2153c9eea648c890218b6c2b53257a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:46.102065 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d46ada-28db-4625-aba2-805b43fc7733 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:46.113857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d46ada-28db-4625-aba2-805b43fc7733 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:46.151537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:46.185457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:46.186347 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:46.186627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:46.186905 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] PATH_INFO: `/v3/users/051a2064ee494218a88b69e9a86c0196` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:46.188114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051a2064ee494218a88b69e9a86c0196)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:46.190107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:46.291498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051a2064ee494218a88b69e9a86c0196'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.308551 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051a2064ee494218a88b69e9a86c0196'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.319232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051a2064ee494218a88b69e9a86c0196'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.319232 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051a2064ee494218a88b69e9a86c0196'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.331028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-d92c5793-4227-4323-a8b0-1a80694be190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051a2064ee494218a88b69e9a86c0196'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.337237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1231/2476] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:46 2022] DELETE /identity/v3/users/051a2064ee494218a88b69e9a86c0196 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:46.778212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:46.779690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:46.794814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2f24d-bced-4c42-b928-89e9fc60f47b admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:46.857906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:46.858812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:46.859043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:46.859314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] PATH_INFO: `/v3/projects/d21d15563f4a4f33bd319c90871d8573` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:46.860501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d21d15563f4a4f33bd319c90871d8573)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:46.862428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:46.912020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd21d15563f4a4f33bd319c90871d8573', 'request_id': 'req-23339c47-fc31-48db-a331-b04d2213462e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.920100 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd21d15563f4a4f33bd319c90871d8573', 'request_id': 'req-23339c47-fc31-48db-a331-b04d2213462e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.926786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd21d15563f4a4f33bd319c90871d8573', 'request_id': 'req-23339c47-fc31-48db-a331-b04d2213462e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.939780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] The token cache is being invalidate because project d21d15563f4a4f33bd319c90871d8573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:46.940158 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.940903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-23339c47-fc31-48db-a331-b04d2213462e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:46.942570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1246/2477] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:46 2022] DELETE /identity/v3/projects/d21d15563f4a4f33bd319c90871d8573 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:46.947518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615765ec-5079-42d6-a44f-7a9992e3c9ea None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:46.947918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615765ec-5079-42d6-a44f-7a9992e3c9ea None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:46.949052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615765ec-5079-42d6-a44f-7a9992e3c9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:46.966783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-615765ec-5079-42d6-a44f-7a9992e3c9ea None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:46.968004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-615765ec-5079-42d6-a44f-7a9992e3c9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:47.027932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1232/2478] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:47.035807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85704421-86a7-4396-9b17-c7519a42adc5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.036099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85704421-86a7-4396-9b17-c7519a42adc5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.036375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85704421-86a7-4396-9b17-c7519a42adc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.055393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-85704421-86a7-4396-9b17-c7519a42adc5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:47.056733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-85704421-86a7-4396-9b17-c7519a42adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:47.065521 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1247/2479] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:47.069740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615765ec-5079-42d6-a44f-7a9992e3c9ea None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.080048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615765ec-5079-42d6-a44f-7a9992e3c9ea None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.084710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd78cf-8fb6-4577-be8f-a136b57cd5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.085375 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd78cf-8fb6-4577-be8f-a136b57cd5cf admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.085658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd78cf-8fb6-4577-be8f-a136b57cd5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.085936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd78cf-8fb6-4577-be8f-a136b57cd5cf admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.086669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd78cf-8fb6-4577-be8f-a136b57cd5cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:55:47.086976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd78cf-8fb6-4577-be8f-a136b57cd5cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.088600 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd78cf-8fb6-4577-be8f-a136b57cd5cf admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.102059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1233/2480] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:55:47.107338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85704421-86a7-4396-9b17-c7519a42adc5 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.116946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85704421-86a7-4396-9b17-c7519a42adc5 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.121917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3c522-b325-470d-b2a7-2002f39c61f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.122571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c522-b325-470d-b2a7-2002f39c61f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.122858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c522-b325-470d-b2a7-2002f39c61f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.123133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3c522-b325-470d-b2a7-2002f39c61f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.123837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3c522-b325-470d-b2a7-2002f39c61f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.125779 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c3c522-b325-470d-b2a7-2002f39c61f5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.153531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1248/2481] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:55:47 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:47.157755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd78cf-8fb6-4577-be8f-a136b57cd5cf admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.168351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd78cf-8fb6-4577-be8f-a136b57cd5cf admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.201958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:47.227549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b9390-eec9-494e-8366-606ba804626d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.228220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b9390-eec9-494e-8366-606ba804626d admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.228505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b9390-eec9-494e-8366-606ba804626d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.228809 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b9390-eec9-494e-8366-606ba804626d admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.230064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b9390-eec9-494e-8366-606ba804626d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.232212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b9390-eec9-494e-8366-606ba804626d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.255638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1234/2482] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:55:47 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:47.257061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3c522-b325-470d-b2a7-2002f39c61f5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.267715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3c522-b325-470d-b2a7-2002f39c61f5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.273035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37dcf-91df-4b69-acee-386dee64569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.273720 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37dcf-91df-4b69-acee-386dee64569d admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.273997 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37dcf-91df-4b69-acee-386dee64569d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.274286 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d37dcf-91df-4b69-acee-386dee64569d admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.275692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d37dcf-91df-4b69-acee-386dee64569d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.278130 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d37dcf-91df-4b69-acee-386dee64569d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.289856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1249/2483] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:47.294236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b9390-eec9-494e-8366-606ba804626d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.304542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b9390-eec9-494e-8366-606ba804626d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.310043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b7869-d2d6-41ac-ae17-ef39b786bebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.310738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b7869-d2d6-41ac-ae17-ef39b786bebe admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.311020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b7869-d2d6-41ac-ae17-ef39b786bebe admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.311297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b7869-d2d6-41ac-ae17-ef39b786bebe admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.312255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b7869-d2d6-41ac-ae17-ef39b786bebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.313932 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b7869-d2d6-41ac-ae17-ef39b786bebe admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.327650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1235/2484] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:47.332359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37dcf-91df-4b69-acee-386dee64569d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.344182 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d37dcf-91df-4b69-acee-386dee64569d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.349573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0053203-d58b-4892-9388-da3f05a98d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.351998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0053203-d58b-4892-9388-da3f05a98d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.352606 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0053203-d58b-4892-9388-da3f05a98d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.352970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0053203-d58b-4892-9388-da3f05a98d69 admin admin] PATH_INFO: `/v3/projects/05d49917b0604a8489699a13d059867c/users/e257f975073d409f98253866f9de5590/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.372068 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0053203-d58b-4892-9388-da3f05a98d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d49917b0604a8489699a13d059867c, user_id=e257f975073d409f98253866f9de5590, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.375249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0053203-d58b-4892-9388-da3f05a98d69 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.389824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:47.411597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1250/2485] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:47 2022] PUT /identity/v3/projects/05d49917b0604a8489699a13d059867c/users/e257f975073d409f98253866f9de5590/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:47.413994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda509a-3842-435d-a921-78807eaead26 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.414479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda509a-3842-435d-a921-78807eaead26 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.414764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda509a-3842-435d-a921-78807eaead26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.443339 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-dfda509a-3842-435d-a921-78807eaead26 None None] MFA Rules not processed for user `e257f975073d409f98253866f9de5590`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:47.444542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-dfda509a-3842-435d-a921-78807eaead26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:47.514108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1236/2486] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:47.518770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0053203-d58b-4892-9388-da3f05a98d69 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.586194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0053203-d58b-4892-9388-da3f05a98d69 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.591262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96692d2c-db51-4fbf-9c2d-1d5c055c789c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.591993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692d2c-db51-4fbf-9c2d-1d5c055c789c admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.592350 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692d2c-db51-4fbf-9c2d-1d5c055c789c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.592738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96692d2c-db51-4fbf-9c2d-1d5c055c789c admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.594135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96692d2c-db51-4fbf-9c2d-1d5c055c789c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.596063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96692d2c-db51-4fbf-9c2d-1d5c055c789c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.631107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1251/2487] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:55:47 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:55:47.633097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda509a-3842-435d-a921-78807eaead26 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.645011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda509a-3842-435d-a921-78807eaead26 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.726463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e5ee5-9eed-4d3d-b9f2-381697f5f1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.727162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e5ee5-9eed-4d3d-b9f2-381697f5f1c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.727452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e5ee5-9eed-4d3d-b9f2-381697f5f1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.727726 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e5ee5-9eed-4d3d-b9f2-381697f5f1c7 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.728460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e5ee5-9eed-4d3d-b9f2-381697f5f1c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.731930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e5ee5-9eed-4d3d-b9f2-381697f5f1c7 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.758081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1237/2488] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:55:47 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:55:47.759573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96692d2c-db51-4fbf-9c2d-1d5c055c789c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.770651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96692d2c-db51-4fbf-9c2d-1d5c055c789c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.775373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe5ef1-7554-4fdd-8fc3-651a3f4fb1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.776030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe5ef1-7554-4fdd-8fc3-651a3f4fb1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.776875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe5ef1-7554-4fdd-8fc3-651a3f4fb1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.777154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe5ef1-7554-4fdd-8fc3-651a3f4fb1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.777882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe5ef1-7554-4fdd-8fc3-651a3f4fb1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.779490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe5ef1-7554-4fdd-8fc3-651a3f4fb1c9 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.790321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1252/2489] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:47.795138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e5ee5-9eed-4d3d-b9f2-381697f5f1c7 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.805121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e5ee5-9eed-4d3d-b9f2-381697f5f1c7 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.809824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3b4a5-ae74-4218-abac-21dcc58a26b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.810715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b4a5-ae74-4218-abac-21dcc58a26b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.810996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b4a5-ae74-4218-abac-21dcc58a26b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.811272 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b4a5-ae74-4218-abac-21dcc58a26b3 admin admin] PATH_INFO: `/v3/projects/15863610cd8e4074a0bc0ec28dd5f5fd/users/ec102e0e01ae489a9f8c3805b3b9c44e/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.826377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3b4a5-ae74-4218-abac-21dcc58a26b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15863610cd8e4074a0bc0ec28dd5f5fd, user_id=ec102e0e01ae489a9f8c3805b3b9c44e, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.829112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3b4a5-ae74-4218-abac-21dcc58a26b3 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.843332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1238/2490] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:47 2022] PUT /identity/v3/projects/15863610cd8e4074a0bc0ec28dd5f5fd/users/ec102e0e01ae489a9f8c3805b3b9c44e/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:47.852362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe5ef1-7554-4fdd-8fc3-651a3f4fb1c9 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.866576 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe5ef1-7554-4fdd-8fc3-651a3f4fb1c9 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.927989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b3e9e-cc5d-4744-bd17-19bc0b5840ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.928820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b3e9e-cc5d-4744-bd17-19bc0b5840ba admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.929115 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b3e9e-cc5d-4744-bd17-19bc0b5840ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.929385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b3e9e-cc5d-4744-bd17-19bc0b5840ba admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.930113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b3e9e-cc5d-4744-bd17-19bc0b5840ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.936536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b3e9e-cc5d-4744-bd17-19bc0b5840ba admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.947420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1253/2491] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:55:47.952654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3b4a5-ae74-4218-abac-21dcc58a26b3 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:47.964276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3b4a5-ae74-4218-abac-21dcc58a26b3 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:47.970956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedf58f-84f6-4757-b74b-0ee30f2b8c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:47.971929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedf58f-84f6-4757-b74b-0ee30f2b8c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:47.972218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedf58f-84f6-4757-b74b-0ee30f2b8c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:47.972500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedf58f-84f6-4757-b74b-0ee30f2b8c9d admin admin] PATH_INFO: `/v3/projects/15863610cd8e4074a0bc0ec28dd5f5fd/users/ec102e0e01ae489a9f8c3805b3b9c44e/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:47.975005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedf58f-84f6-4757-b74b-0ee30f2b8c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=15863610cd8e4074a0bc0ec28dd5f5fd, user_id=ec102e0e01ae489a9f8c3805b3b9c44e, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:47.978895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedf58f-84f6-4757-b74b-0ee30f2b8c9d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:47.984919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:47.990901 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1239/2492] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:55:47 2022] PUT /identity/v3/projects/15863610cd8e4074a0bc0ec28dd5f5fd/users/ec102e0e01ae489a9f8c3805b3b9c44e/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:48.057363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e2af4-e44d-4976-9308-e43803eb444a None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:48.058134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e2af4-e44d-4976-9308-e43803eb444a None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:48.058551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e2af4-e44d-4976-9308-e43803eb444a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:48.086090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d51e2af4-e44d-4976-9308-e43803eb444a None None] MFA Rules not processed for user `ec102e0e01ae489a9f8c3805b3b9c44e`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:48.087311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d51e2af4-e44d-4976-9308-e43803eb444a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:48.147475 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:48.157901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1254/2493] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:48.168482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedf58f-84f6-4757-b74b-0ee30f2b8c9d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:48.170281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-adedf58f-84f6-4757-b74b-0ee30f2b8c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:48.226332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedf58f-84f6-4757-b74b-0ee30f2b8c9d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:48.230961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9377288-28e6-40da-8114-f46e82cf7026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:48.231595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9377288-28e6-40da-8114-f46e82cf7026 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:48.231876 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9377288-28e6-40da-8114-f46e82cf7026 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:48.232152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9377288-28e6-40da-8114-f46e82cf7026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:48.286651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9377288-28e6-40da-8114-f46e82cf7026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:48.288081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9377288-28e6-40da-8114-f46e82cf7026 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:48.297552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1240/2494] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:52.681928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e2af4-e44d-4976-9308-e43803eb444a None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:52.683756 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d51e2af4-e44d-4976-9308-e43803eb444a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:52.696549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e2af4-e44d-4976-9308-e43803eb444a None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:52.701078 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc289bce-9423-480c-83d9-8416820ad2ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:52.701746 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc289bce-9423-480c-83d9-8416820ad2ec service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:52.702034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc289bce-9423-480c-83d9-8416820ad2ec service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:52.702316 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc289bce-9423-480c-83d9-8416820ad2ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:52.712096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc289bce-9423-480c-83d9-8416820ad2ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:52.713591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc289bce-9423-480c-83d9-8416820ad2ec service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:52.724194 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1255/2495] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:54.158590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9377288-28e6-40da-8114-f46e82cf7026 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:54.159995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e9377288-28e6-40da-8114-f46e82cf7026 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:54.224898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9377288-28e6-40da-8114-f46e82cf7026 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:54.227872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabdea4-8074-41d0-bcc4-e5a87cabadee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:54.228533 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabdea4-8074-41d0-bcc4-e5a87cabadee service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:54.228853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabdea4-8074-41d0-bcc4-e5a87cabadee service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:54.229134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabdea4-8074-41d0-bcc4-e5a87cabadee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:54.230987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5aabdea4-8074-41d0-bcc4-e5a87cabadee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:54.317167 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabdea4-8074-41d0-bcc4-e5a87cabadee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:54.318953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabdea4-8074-41d0-bcc4-e5a87cabadee service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:54.332395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1241/2496] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:54.359644 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc289bce-9423-480c-83d9-8416820ad2ec service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:54.361172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-dc289bce-9423-480c-83d9-8416820ad2ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:54.373796 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc289bce-9423-480c-83d9-8416820ad2ec service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:54.377698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b772987-8630-4543-b8d2-c1cd87db0b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:54.378619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b772987-8630-4543-b8d2-c1cd87db0b01 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:54.379102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b772987-8630-4543-b8d2-c1cd87db0b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:54.379494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b772987-8630-4543-b8d2-c1cd87db0b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:54.381362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-6b772987-8630-4543-b8d2-c1cd87db0b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:54.395852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b772987-8630-4543-b8d2-c1cd87db0b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:54.397425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b772987-8630-4543-b8d2-c1cd87db0b01 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:54.408666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1256/2497] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:57.706668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabdea4-8074-41d0-bcc4-e5a87cabadee service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:57.711708 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:57.718283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabdea4-8074-41d0-bcc4-e5a87cabadee service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:57.723030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:57.723938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:57.724218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:57.724487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] PATH_INFO: `/v3/users/e257f975073d409f98253866f9de5590` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:57.725699 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e257f975073d409f98253866f9de5590)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:57.727612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:57.776301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e257f975073d409f98253866f9de5590'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:57.789662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e257f975073d409f98253866f9de5590'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:57.797486 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e257f975073d409f98253866f9de5590'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:57.798095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e257f975073d409f98253866f9de5590'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:57.805619 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e257f975073d409f98253866f9de5590'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:57.812384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1242/2498] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:57 2022] DELETE /identity/v3/users/e257f975073d409f98253866f9de5590 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:57.817247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6df784-79f4-41c1-a7ad-afe53f3f0d1c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:57.817827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6df784-79f4-41c1-a7ad-afe53f3f0d1c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:57.817923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6df784-79f4-41c1-a7ad-afe53f3f0d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:57.836824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2b6df784-79f4-41c1-a7ad-afe53f3f0d1c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:55:57.839468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2b6df784-79f4-41c1-a7ad-afe53f3f0d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:57.978666 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1257/2499] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:55:57.994149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:57.996218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:58.079958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8637908-1ad3-4f7c-90fe-7c1de0bea591 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:58.083844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a33e-b2cd-4948-98bb-7f1219a7c654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:58.084650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a33e-b2cd-4948-98bb-7f1219a7c654 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:58.085282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a33e-b2cd-4948-98bb-7f1219a7c654 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:58.085282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a33e-b2cd-4948-98bb-7f1219a7c654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:58.095691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5a33e-b2cd-4948-98bb-7f1219a7c654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:58.097366 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5a33e-b2cd-4948-98bb-7f1219a7c654 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:58.109788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1243/2500] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:58.510486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6df784-79f4-41c1-a7ad-afe53f3f0d1c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:58.512843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2b6df784-79f4-41c1-a7ad-afe53f3f0d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:58.529935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6df784-79f4-41c1-a7ad-afe53f3f0d1c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:58.534524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:58.535406 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:58.535692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:58.535973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] PATH_INFO: `/v3/projects/05d49917b0604a8489699a13d059867c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:58.537217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d49917b0604a8489699a13d059867c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:58.539280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:58.595236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d49917b0604a8489699a13d059867c', 'request_id': 'req-c1b6be31-ccca-4dce-8109-287894eedb19'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:58.604864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d49917b0604a8489699a13d059867c', 'request_id': 'req-c1b6be31-ccca-4dce-8109-287894eedb19'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:58.611492 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d49917b0604a8489699a13d059867c', 'request_id': 'req-c1b6be31-ccca-4dce-8109-287894eedb19'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:58.624933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] The token cache is being invalidate because project 05d49917b0604a8489699a13d059867c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:58.625231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:58.625901 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:58.627550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1258/2501] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:58 2022] DELETE /identity/v3/projects/05d49917b0604a8489699a13d059867c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:58.631423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a33e-b2cd-4948-98bb-7f1219a7c654 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:58.642527 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a33e-b2cd-4948-98bb-7f1219a7c654 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:58.851427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:58.852431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:58.852861 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:58.853230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] PATH_INFO: `/v3/users/ec102e0e01ae489a9f8c3805b3b9c44e` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:58.860915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec102e0e01ae489a9f8c3805b3b9c44e)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:58.863051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:58.915247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:58.989992 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec102e0e01ae489a9f8c3805b3b9c44e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.010252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec102e0e01ae489a9f8c3805b3b9c44e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.029306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec102e0e01ae489a9f8c3805b3b9c44e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.031063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec102e0e01ae489a9f8c3805b3b9c44e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.041338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec102e0e01ae489a9f8c3805b3b9c44e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.053107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1244/2502] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:58 2022] DELETE /identity/v3/users/ec102e0e01ae489a9f8c3805b3b9c44e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:59.216849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:59.219482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:59.387098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:59.409668 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6be31-ccca-4dce-8109-287894eedb19 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:59.455474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:59.457116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:59.504389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:59.505396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:59.505750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:59.506135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:59.508542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:59.525610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:59.529801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:59.539060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1259/2503] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:55:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:55:59.539825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:59.549778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:59.565873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e0b02-c022-46e7-b4ce-e403297ff1cf service placement] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:59.570768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:59.571424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9d63b-ad6c-497f-b78a-b7670e7026ba admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:59.571631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:59.572042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:59.572321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] PATH_INFO: `/v3/users/04e35f840ef045f0b44fab3e57a1a3c8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:59.574657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e35f840ef045f0b44fab3e57a1a3c8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:59.576610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:59.578338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:59.579330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:59.579690 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:59.580062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] PATH_INFO: `/v3/users/927ffbcd7078473f97f42bdb54acf57e` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:59.581437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=927ffbcd7078473f97f42bdb54acf57e)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:59.583464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:59.630958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e35f840ef045f0b44fab3e57a1a3c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.634265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927ffbcd7078473f97f42bdb54acf57e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.645250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e35f840ef045f0b44fab3e57a1a3c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.646036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e35f840ef045f0b44fab3e57a1a3c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.646798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e35f840ef045f0b44fab3e57a1a3c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.648027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927ffbcd7078473f97f42bdb54acf57e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.655250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e35f840ef045f0b44fab3e57a1a3c8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.657762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927ffbcd7078473f97f42bdb54acf57e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.658406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927ffbcd7078473f97f42bdb54acf57e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.665530 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927ffbcd7078473f97f42bdb54acf57e'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.668055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1260/2504] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:59 2022] DELETE /identity/v3/users/04e35f840ef045f0b44fab3e57a1a3c8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:59.668961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:59.672265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:59.675599 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1245/2505] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:55:59 2022] DELETE /identity/v3/users/927ffbcd7078473f97f42bdb54acf57e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:55:59.753464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:59.758187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d0895-409f-4cb0-89ce-4da12e6cfc4d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:59.835452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:59.836672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:59.837024 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:59.837527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] PATH_INFO: `/v3/projects/15863610cd8e4074a0bc0ec28dd5f5fd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:59.839581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15863610cd8e4074a0bc0ec28dd5f5fd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:59.841632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:55:59.878703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:59.880710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:59.899276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c0e4-29be-43cc-9051-8b86b1d0cc05 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:55:59.906875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15863610cd8e4074a0bc0ec28dd5f5fd', 'request_id': 'req-908e6501-5b2f-4b45-bdc2-0ea80730b950'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.914456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:55:59.916290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15863610cd8e4074a0bc0ec28dd5f5fd', 'request_id': 'req-908e6501-5b2f-4b45-bdc2-0ea80730b950'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.925949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15863610cd8e4074a0bc0ec28dd5f5fd', 'request_id': 'req-908e6501-5b2f-4b45-bdc2-0ea80730b950'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.941206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] The token cache is being invalidate because project 15863610cd8e4074a0bc0ec28dd5f5fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:55:59.941650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.942744 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:55:59.948642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1261/2506] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:59 2022] DELETE /identity/v3/projects/15863610cd8e4074a0bc0ec28dd5f5fd => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:55:59.948642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:55:59.948642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:55:59.978430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:55:59.979352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:55:59.979632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:55:59.979961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] PATH_INFO: `/v3/projects/a779b000010c4549989d8a8f25e8b1be` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:55:59.981233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=a779b000010c4549989d8a8f25e8b1be)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:55:59.983092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.017622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e6501-5b2f-4b45-bdc2-0ea80730b950 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.023834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.025080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.025817 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.026764 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] PATH_INFO: `/v3/projects/040134c568aa46c09ca9e5b077cc5d3c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.028107 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=040134c568aa46c09ca9e5b077cc5d3c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.030582 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.035575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a779b000010c4549989d8a8f25e8b1be', 'request_id': 'req-95f9b658-6551-4fcf-98a7-bcb925c512ff'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.043068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a779b000010c4549989d8a8f25e8b1be', 'request_id': 'req-95f9b658-6551-4fcf-98a7-bcb925c512ff'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.051853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a779b000010c4549989d8a8f25e8b1be', 'request_id': 'req-95f9b658-6551-4fcf-98a7-bcb925c512ff'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.064934 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] The token cache is being invalidate because project a779b000010c4549989d8a8f25e8b1be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:00.065326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.066134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-95f9b658-6551-4fcf-98a7-bcb925c512ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.068175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1246/2507] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:59 2022] DELETE /identity/v3/projects/a779b000010c4549989d8a8f25e8b1be => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:00.069832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bfe22-456e-4625-baba-478cfe40363d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.070201 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bfe22-456e-4625-baba-478cfe40363d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.070487 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bfe22-456e-4625-baba-478cfe40363d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.085738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:00.098916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '040134c568aa46c09ca9e5b077cc5d3c', 'request_id': 'req-24922917-a4d1-4372-b851-8e399c586581'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.099848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-098bfe22-456e-4625-baba-478cfe40363d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:00.101293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-098bfe22-456e-4625-baba-478cfe40363d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:00.110882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '040134c568aa46c09ca9e5b077cc5d3c', 'request_id': 'req-24922917-a4d1-4372-b851-8e399c586581'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.119500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '040134c568aa46c09ca9e5b077cc5d3c', 'request_id': 'req-24922917-a4d1-4372-b851-8e399c586581'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.134691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] The token cache is being invalidate because project 040134c568aa46c09ca9e5b077cc5d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:00.134988 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.137358 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-24922917-a4d1-4372-b851-8e399c586581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:00.139105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1262/2508] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:55:59 2022] DELETE /identity/v3/projects/040134c568aa46c09ca9e5b077cc5d3c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:00.140446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e26ad-647c-4a17-bf06-8bc34378c1a4 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.140793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e26ad-647c-4a17-bf06-8bc34378c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.141079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e26ad-647c-4a17-bf06-8bc34378c1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.165499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-ca2e26ad-647c-4a17-bf06-8bc34378c1a4 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:00.168639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ca2e26ad-647c-4a17-bf06-8bc34378c1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:00.175197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1247/2509] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:00.176337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e0160-f4ed-47f3-bd84-4e5b1b6fe0b1 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.176636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e0160-f4ed-47f3-bd84-4e5b1b6fe0b1 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.176920 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e0160-f4ed-47f3-bd84-4e5b1b6fe0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.184615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1263/2510] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:00.186246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656aaf8-9465-4038-90a0-c48f01b2b138 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.186586 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656aaf8-9465-4038-90a0-c48f01b2b138 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.186874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656aaf8-9465-4038-90a0-c48f01b2b138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.205249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-077e0160-f4ed-47f3-bd84-4e5b1b6fe0b1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:00.206473 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-077e0160-f4ed-47f3-bd84-4e5b1b6fe0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:00.207590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d656aaf8-9465-4038-90a0-c48f01b2b138 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:00.208492 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d656aaf8-9465-4038-90a0-c48f01b2b138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:00.216307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1248/2511] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:00.217520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1264/2512] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:00.220669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3775a3-ccb4-42de-87d3-2565c1a677c0 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.220904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3775a3-ccb4-42de-87d3-2565c1a677c0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.221191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3775a3-ccb4-42de-87d3-2565c1a677c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.221323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656aaf8-9465-4038-90a0-c48f01b2b138 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.232177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656aaf8-9465-4038-90a0-c48f01b2b138 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.237254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2de49-bb88-4c5c-8ba4-445d03bb4586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.237940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2de49-bb88-4c5c-8ba4-445d03bb4586 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.239373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2de49-bb88-4c5c-8ba4-445d03bb4586 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.239373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2de49-bb88-4c5c-8ba4-445d03bb4586 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.239373 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2de49-bb88-4c5c-8ba4-445d03bb4586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:00.239662 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2de49-bb88-4c5c-8ba4-445d03bb4586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.241143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2de49-bb88-4c5c-8ba4-445d03bb4586 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.242109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-df3775a3-ccb4-42de-87d3-2565c1a677c0 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:00.243616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-df3775a3-ccb4-42de-87d3-2565c1a677c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:00.252534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1249/2513] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:00.257142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1265/2514] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:56:00.257302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c3a30-ef8e-46db-82a7-d9019caed520 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.257302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c3a30-ef8e-46db-82a7-d9019caed520 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.257302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c3a30-ef8e-46db-82a7-d9019caed520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.257582 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2de49-bb88-4c5c-8ba4-445d03bb4586 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.267386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2de49-bb88-4c5c-8ba4-445d03bb4586 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.272827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693edec9-6830-4b49-8adb-2023f534f69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.273441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693edec9-6830-4b49-8adb-2023f534f69e admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.273691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-cb2c3a30-ef8e-46db-82a7-d9019caed520 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:00.273829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693edec9-6830-4b49-8adb-2023f534f69e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.273996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693edec9-6830-4b49-8adb-2023f534f69e admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.274749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693edec9-6830-4b49-8adb-2023f534f69e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:00.274865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-cb2c3a30-ef8e-46db-82a7-d9019caed520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:00.275093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693edec9-6830-4b49-8adb-2023f534f69e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.277413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693edec9-6830-4b49-8adb-2023f534f69e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.283545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1250/2515] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:00.284134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c3a30-ef8e-46db-82a7-d9019caed520 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.290567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1266/2516] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:56:00.291419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693edec9-6830-4b49-8adb-2023f534f69e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.302865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c3a30-ef8e-46db-82a7-d9019caed520 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.303196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693edec9-6830-4b49-8adb-2023f534f69e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.308142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0896b-bfd9-4bce-94c8-1d709ede1aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.308736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df072e0-deeb-476c-8045-8b9327fdd6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.309375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0896b-bfd9-4bce-94c8-1d709ede1aed admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.309501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df072e0-deeb-476c-8045-8b9327fdd6fc admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.309659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0896b-bfd9-4bce-94c8-1d709ede1aed admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.309769 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df072e0-deeb-476c-8045-8b9327fdd6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.309946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0896b-bfd9-4bce-94c8-1d709ede1aed admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.310173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df072e0-deeb-476c-8045-8b9327fdd6fc admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.310632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0896b-bfd9-4bce-94c8-1d709ede1aed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:00.310888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df072e0-deeb-476c-8045-8b9327fdd6fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.310989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0896b-bfd9-4bce-94c8-1d709ede1aed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.312512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0896b-bfd9-4bce-94c8-1d709ede1aed admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.313061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df072e0-deeb-476c-8045-8b9327fdd6fc admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.328703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1267/2517] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:00.328703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0896b-bfd9-4bce-94c8-1d709ede1aed admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.340246 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0896b-bfd9-4bce-94c8-1d709ede1aed admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.345153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798dbb9-6665-4540-96e4-d5b15d419560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.345880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798dbb9-6665-4540-96e4-d5b15d419560 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.346083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798dbb9-6665-4540-96e4-d5b15d419560 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.346637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798dbb9-6665-4540-96e4-d5b15d419560 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.347433 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798dbb9-6665-4540-96e4-d5b15d419560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.348806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1251/2518] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:00.349383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798dbb9-6665-4540-96e4-d5b15d419560 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.350217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df072e0-deeb-476c-8045-8b9327fdd6fc admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.356438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:00.380142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1268/2519] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:00.385287 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798dbb9-6665-4540-96e4-d5b15d419560 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.425718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df072e0-deeb-476c-8045-8b9327fdd6fc admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.460244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798dbb9-6665-4540-96e4-d5b15d419560 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.468231 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbee949-adf3-4eda-b5fd-93e13b9c1be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.469053 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbee949-adf3-4eda-b5fd-93e13b9c1be2 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.469610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbee949-adf3-4eda-b5fd-93e13b9c1be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.469963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbee949-adf3-4eda-b5fd-93e13b9c1be2 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.470741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbee949-adf3-4eda-b5fd-93e13b9c1be2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.473410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbee949-adf3-4eda-b5fd-93e13b9c1be2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.491388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcba681-d1ae-4f51-b637-d74eb0078dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.492050 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcba681-d1ae-4f51-b637-d74eb0078dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.492448 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcba681-d1ae-4f51-b637-d74eb0078dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.492768 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcba681-d1ae-4f51-b637-d74eb0078dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.493771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcba681-d1ae-4f51-b637-d74eb0078dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.496247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcba681-d1ae-4f51-b637-d74eb0078dd8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.507354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1269/2520] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:00.508190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbee949-adf3-4eda-b5fd-93e13b9c1be2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.521622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbee949-adf3-4eda-b5fd-93e13b9c1be2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.528032 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575e9b6-4d34-4e00-be8a-7174a31b28a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.528914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575e9b6-4d34-4e00-be8a-7174a31b28a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.529264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575e9b6-4d34-4e00-be8a-7174a31b28a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.529612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575e9b6-4d34-4e00-be8a-7174a31b28a4 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.530402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575e9b6-4d34-4e00-be8a-7174a31b28a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.533505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575e9b6-4d34-4e00-be8a-7174a31b28a4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.538326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1252/2521] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/projects => generated 405 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:00.542897 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcba681-d1ae-4f51-b637-d74eb0078dd8 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.559924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1270/2522] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/users => generated 287 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:00.561105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575e9b6-4d34-4e00-be8a-7174a31b28a4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.571592 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575e9b6-4d34-4e00-be8a-7174a31b28a4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.597413 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=88,keystone:DELETE=4,keystone:INSERT=8 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:00.620651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcba681-d1ae-4f51-b637-d74eb0078dd8 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.625883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71199543-5fb3-415d-b399-e296a70435e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.626626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71199543-5fb3-415d-b399-e296a70435e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.626844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71199543-5fb3-415d-b399-e296a70435e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.627616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71199543-5fb3-415d-b399-e296a70435e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.627868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71199543-5fb3-415d-b399-e296a70435e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.629496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71199543-5fb3-415d-b399-e296a70435e3 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.643953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1253/2523] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:00.643953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71199543-5fb3-415d-b399-e296a70435e3 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.645417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295820a-b777-4509-9822-7151ff5a9346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.646105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295820a-b777-4509-9822-7151ff5a9346 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.646394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295820a-b777-4509-9822-7151ff5a9346 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.646672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295820a-b777-4509-9822-7151ff5a9346 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.647370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295820a-b777-4509-9822-7151ff5a9346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.649513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295820a-b777-4509-9822-7151ff5a9346 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.655053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71199543-5fb3-415d-b399-e296a70435e3 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.661196 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10398b0-ea83-4b79-b1bd-0820f9bb0cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.661998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10398b0-ea83-4b79-b1bd-0820f9bb0cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.662480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10398b0-ea83-4b79-b1bd-0820f9bb0cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.662764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10398b0-ea83-4b79-b1bd-0820f9bb0cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.663489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10398b0-ea83-4b79-b1bd-0820f9bb0cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.665328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10398b0-ea83-4b79-b1bd-0820f9bb0cf1 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.678029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1271/2524] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/users => generated 303 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:00.678029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295820a-b777-4509-9822-7151ff5a9346 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.684889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1254/2525] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:00.684889 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10398b0-ea83-4b79-b1bd-0820f9bb0cf1 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.689446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295820a-b777-4509-9822-7151ff5a9346 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.694834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10398b0-ea83-4b79-b1bd-0820f9bb0cf1 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.695132 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3cf01-4414-489f-8e45-19d91f608ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.696127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3cf01-4414-489f-8e45-19d91f608ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.696127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3cf01-4414-489f-8e45-19d91f608ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.696548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3cf01-4414-489f-8e45-19d91f608ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.697387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3cf01-4414-489f-8e45-19d91f608ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.699752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3cf01-4414-489f-8e45-19d91f608ed1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.701398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e05bc1-e956-4e6c-8934-f1f2ebf56ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.702360 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e05bc1-e956-4e6c-8934-f1f2ebf56ace admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.702841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e05bc1-e956-4e6c-8934-f1f2ebf56ace admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.703252 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e05bc1-e956-4e6c-8934-f1f2ebf56ace admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.704085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e05bc1-e956-4e6c-8934-f1f2ebf56ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.705786 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e05bc1-e956-4e6c-8934-f1f2ebf56ace admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.711344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1272/2526] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:00.711982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3cf01-4414-489f-8e45-19d91f608ed1 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.722615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1255/2527] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:00.722615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e05bc1-e956-4e6c-8934-f1f2ebf56ace admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.728512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3cf01-4414-489f-8e45-19d91f608ed1 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.731112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e05bc1-e956-4e6c-8934-f1f2ebf56ace admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.733855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e187a-8061-44cd-8fe3-4b48335ae4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.734709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e187a-8061-44cd-8fe3-4b48335ae4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.735342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e187a-8061-44cd-8fe3-4b48335ae4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.735702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e187a-8061-44cd-8fe3-4b48335ae4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.736071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59090-f14b-48a1-a4cb-94d2b9929cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.736593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e187a-8061-44cd-8fe3-4b48335ae4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.737001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59090-f14b-48a1-a4cb-94d2b9929cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.737283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59090-f14b-48a1-a4cb-94d2b9929cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.737563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59090-f14b-48a1-a4cb-94d2b9929cc9 admin admin] PATH_INFO: `/v3/projects/40fd86bfacae4cb6adb22f54962238d6/users/fdd3aa5da8a74c0a85d3363dc211257d/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.738508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e187a-8061-44cd-8fe3-4b48335ae4b5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.750434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:00.752550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1273/2528] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:00.753579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e187a-8061-44cd-8fe3-4b48335ae4b5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.761662 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a59090-f14b-48a1-a4cb-94d2b9929cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fd86bfacae4cb6adb22f54962238d6, user_id=fdd3aa5da8a74c0a85d3363dc211257d, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.764803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a59090-f14b-48a1-a4cb-94d2b9929cc9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.765112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e187a-8061-44cd-8fe3-4b48335ae4b5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.771114 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf62374-5cd8-42ed-959f-18037d53784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.771886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf62374-5cd8-42ed-959f-18037d53784e admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.772218 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf62374-5cd8-42ed-959f-18037d53784e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.772560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf62374-5cd8-42ed-959f-18037d53784e admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.773472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf62374-5cd8-42ed-959f-18037d53784e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.775220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf62374-5cd8-42ed-959f-18037d53784e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.784006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1256/2529] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:00 2022] PUT /identity/v3/projects/40fd86bfacae4cb6adb22f54962238d6/users/fdd3aa5da8a74c0a85d3363dc211257d/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:00.784862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59090-f14b-48a1-a4cb-94d2b9929cc9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.789595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1274/2530] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:00.796337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf62374-5cd8-42ed-959f-18037d53784e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:00.806477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf62374-5cd8-42ed-959f-18037d53784e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.858174 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59090-f14b-48a1-a4cb-94d2b9929cc9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:00.864241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5353a91-1d56-458b-8947-a4621745ffb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.865923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5353a91-1d56-458b-8947-a4621745ffb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.866311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5353a91-1d56-458b-8947-a4621745ffb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.866742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5353a91-1d56-458b-8947-a4621745ffb4 admin admin] PATH_INFO: `/v3/projects/ac4f723b896943248837d999142f413b/users/e70c8b99aa814154acff411cf355a3bb/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.871567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c5df9-6029-4c41-ae5d-085c1e0e9aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:00.872567 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5df9-6029-4c41-ae5d-085c1e0e9aac admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.873336 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5df9-6029-4c41-ae5d-085c1e0e9aac admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.873836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5df9-6029-4c41-ae5d-085c1e0e9aac admin admin] PATH_INFO: `/v3/projects/03f4862035d6419e84399d63bb8ff59a/users/9024325cab0a4109819d440713e9d8d5/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.886020 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5353a91-1d56-458b-8947-a4621745ffb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4f723b896943248837d999142f413b, user_id=e70c8b99aa814154acff411cf355a3bb, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.888545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5353a91-1d56-458b-8947-a4621745ffb4 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.901083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c5df9-6029-4c41-ae5d-085c1e0e9aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f4862035d6419e84399d63bb8ff59a, user_id=9024325cab0a4109819d440713e9d8d5, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:00.904380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c5df9-6029-4c41-ae5d-085c1e0e9aac admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:00.912995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1257/2531] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:00 2022] PUT /identity/v3/projects/ac4f723b896943248837d999142f413b/users/e70c8b99aa814154acff411cf355a3bb/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:00.912995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179e943-2904-4b81-a7a1-a5261763e421 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.912995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179e943-2904-4b81-a7a1-a5261763e421 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.912995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179e943-2904-4b81-a7a1-a5261763e421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.935645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1275/2532] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:00 2022] PUT /identity/v3/projects/03f4862035d6419e84399d63bb8ff59a/users/9024325cab0a4109819d440713e9d8d5/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:00.935645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a5ef-b2a3-4b75-b020-cf416421435f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:00.935645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a5ef-b2a3-4b75-b020-cf416421435f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:00.935645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a5ef-b2a3-4b75-b020-cf416421435f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:00.946325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-e179e943-2904-4b81-a7a1-a5261763e421 None None] MFA Rules not processed for user `fdd3aa5da8a74c0a85d3363dc211257d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:00.947537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e179e943-2904-4b81-a7a1-a5261763e421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:00.965708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-4d62a5ef-b2a3-4b75-b020-cf416421435f None None] MFA Rules not processed for user `e70c8b99aa814154acff411cf355a3bb`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:00.967408 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4d62a5ef-b2a3-4b75-b020-cf416421435f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:01.020765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1258/2533] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:01.022687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154ed03-f254-477e-8cc6-1ca6fa742c3d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.022979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154ed03-f254-477e-8cc6-1ca6fa742c3d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.023262 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154ed03-f254-477e-8cc6-1ca6fa742c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.029691 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:01.053537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-2154ed03-f254-477e-8cc6-1ca6fa742c3d None None] MFA Rules not processed for user `9024325cab0a4109819d440713e9d8d5`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:01.054005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1276/2534] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:01.054779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2154ed03-f254-477e-8cc6-1ca6fa742c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:01.054910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a5ef-b2a3-4b75-b020-cf416421435f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:01.056488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-4d62a5ef-b2a3-4b75-b020-cf416421435f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:01.123807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:01.127199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a5ef-b2a3-4b75-b020-cf416421435f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:01.131079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e382-b795-45dc-a56d-fc23567c9a1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:01.131755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e382-b795-45dc-a56d-fc23567c9a1e service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.132062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e382-b795-45dc-a56d-fc23567c9a1e service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.132374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e382-b795-45dc-a56d-fc23567c9a1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.143599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92e382-b795-45dc-a56d-fc23567c9a1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:01.145100 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92e382-b795-45dc-a56d-fc23567c9a1e service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:01.147684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1259/2535] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:01 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:01.151459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154ed03-f254-477e-8cc6-1ca6fa742c3d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:01.151459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2154ed03-f254-477e-8cc6-1ca6fa742c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:01.155674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1277/2536] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:01.156486 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e382-b795-45dc-a56d-fc23567c9a1e service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:01.240533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e382-b795-45dc-a56d-fc23567c9a1e service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:01.246422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20094152-816d-4acf-9bcf-3a293c3830bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:01.247081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094152-816d-4acf-9bcf-3a293c3830bc admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.247363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094152-816d-4acf-9bcf-3a293c3830bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.247638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20094152-816d-4acf-9bcf-3a293c3830bc admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.248326 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20094152-816d-4acf-9bcf-3a293c3830bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:01.250258 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20094152-816d-4acf-9bcf-3a293c3830bc admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:01.258331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154ed03-f254-477e-8cc6-1ca6fa742c3d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:01.266467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5e591-a92a-49fb-a3ef-d1f0712db4b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:01.267293 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5e591-a92a-49fb-a3ef-d1f0712db4b0 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.267577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5e591-a92a-49fb-a3ef-d1f0712db4b0 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.268186 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5e591-a92a-49fb-a3ef-d1f0712db4b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.279528 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5e591-a92a-49fb-a3ef-d1f0712db4b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:01.281607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5e591-a92a-49fb-a3ef-d1f0712db4b0 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:01.288412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1278/2537] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:01 2022] POST /identity/v3/projects => generated 405 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:01.289430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20094152-816d-4acf-9bcf-3a293c3830bc admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:01.293618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1260/2538] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:01.387542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20094152-816d-4acf-9bcf-3a293c3830bc admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:01.400359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efa5a7-19ad-462d-9d9a-54084ee58abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:01.401213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efa5a7-19ad-462d-9d9a-54084ee58abc admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.401524 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efa5a7-19ad-462d-9d9a-54084ee58abc admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.401824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efa5a7-19ad-462d-9d9a-54084ee58abc admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.402597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efa5a7-19ad-462d-9d9a-54084ee58abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:01.404574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efa5a7-19ad-462d-9d9a-54084ee58abc admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:01.472642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1279/2539] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:01 2022] POST /identity/v3/users => generated 309 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:01.477749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5e591-a92a-49fb-a3ef-d1f0712db4b0 service nova] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:01.491972 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5e591-a92a-49fb-a3ef-d1f0712db4b0 service nova] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:01.504421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ea271-2fd4-4a99-b057-c1b36b622186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:01.504421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ea271-2fd4-4a99-b057-c1b36b622186 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.504421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ea271-2fd4-4a99-b057-c1b36b622186 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.504421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ea271-2fd4-4a99-b057-c1b36b622186 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.517277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ea271-2fd4-4a99-b057-c1b36b622186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:01.523979 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ea271-2fd4-4a99-b057-c1b36b622186 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:01.555633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1261/2540] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:01.556867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efa5a7-19ad-462d-9d9a-54084ee58abc admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:01.573930 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efa5a7-19ad-462d-9d9a-54084ee58abc admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:01.595098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a05cf-9381-4a2d-91bb-4d8dfc67a6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:01.596066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a05cf-9381-4a2d-91bb-4d8dfc67a6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.596272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a05cf-9381-4a2d-91bb-4d8dfc67a6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.596549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a05cf-9381-4a2d-91bb-4d8dfc67a6b4 admin admin] PATH_INFO: `/v3/projects/ffdc945caf1f4526a7244f733d54234c/users/7aaf0e53c6a744d7a48f5ecce890f6f1/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.600086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:01.648282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a05cf-9381-4a2d-91bb-4d8dfc67a6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdc945caf1f4526a7244f733d54234c, user_id=7aaf0e53c6a744d7a48f5ecce890f6f1, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:01.666058 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a05cf-9381-4a2d-91bb-4d8dfc67a6b4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:01.713198 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1280/2541] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:01 2022] PUT /identity/v3/projects/ffdc945caf1f4526a7244f733d54234c/users/7aaf0e53c6a744d7a48f5ecce890f6f1/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:01.725607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ea271-2fd4-4a99-b057-c1b36b622186 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:01.726992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a05cf-9381-4a2d-91bb-4d8dfc67a6b4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:01.746787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ea271-2fd4-4a99-b057-c1b36b622186 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:01.848888 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:01.849780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.850086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.850697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] PATH_INFO: `/v3/users/e70c8b99aa814154acff411cf355a3bb` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.852980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70c8b99aa814154acff411cf355a3bb)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:01.863711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a05cf-9381-4a2d-91bb-4d8dfc67a6b4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:01.868119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:01.870480 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea016752-b265-4496-a205-88bfda58c5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:01.871731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea016752-b265-4496-a205-88bfda58c5ef admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.872010 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea016752-b265-4496-a205-88bfda58c5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.872525 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea016752-b265-4496-a205-88bfda58c5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.873311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea016752-b265-4496-a205-88bfda58c5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:01.875530 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea016752-b265-4496-a205-88bfda58c5ef admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:01.889423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1281/2542] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:01.894005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea016752-b265-4496-a205-88bfda58c5ef admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:01.906695 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea016752-b265-4496-a205-88bfda58c5ef admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:01.912038 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791632b-13bf-4b0a-b5d4-173152997526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:01.912972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791632b-13bf-4b0a-b5d4-173152997526 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.913257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791632b-13bf-4b0a-b5d4-173152997526 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.913542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791632b-13bf-4b0a-b5d4-173152997526 admin admin] PATH_INFO: `/v3/projects/ffdc945caf1f4526a7244f733d54234c/users/7aaf0e53c6a744d7a48f5ecce890f6f1/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.915568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791632b-13bf-4b0a-b5d4-173152997526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdc945caf1f4526a7244f733d54234c, user_id=7aaf0e53c6a744d7a48f5ecce890f6f1, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:01.919000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791632b-13bf-4b0a-b5d4-173152997526 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:01.931109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70c8b99aa814154acff411cf355a3bb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:01.931629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1282/2543] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:01 2022] PUT /identity/v3/projects/ffdc945caf1f4526a7244f733d54234c/users/7aaf0e53c6a744d7a48f5ecce890f6f1/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:01.943654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeafa7-c20d-48fc-a6d6-858eeadde347 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.943966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeafa7-c20d-48fc-a6d6-858eeadde347 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.944217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeafa7-c20d-48fc-a6d6-858eeadde347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:01.951357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70c8b99aa814154acff411cf355a3bb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:01.953354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:01.964658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70c8b99aa814154acff411cf355a3bb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:01.965740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70c8b99aa814154acff411cf355a3bb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:01.979316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-dcd323b1-214c-47ec-99da-3ca07fbcf03b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70c8b99aa814154acff411cf355a3bb'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:01.986230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-8eeeafa7-c20d-48fc-a6d6-858eeadde347 None None] MFA Rules not processed for user `7aaf0e53c6a744d7a48f5ecce890f6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:01.987711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8eeeafa7-c20d-48fc-a6d6-858eeadde347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:01.988267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1262/2544] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:01 2022] DELETE /identity/v3/users/e70c8b99aa814154acff411cf355a3bb => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:01.992384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ef8ab-405c-4c5b-96c8-b15eca5574af None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:01.993647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ef8ab-405c-4c5b-96c8-b15eca5574af None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:01.993991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ef8ab-405c-4c5b-96c8-b15eca5574af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:02.016352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-425ef8ab-405c-4c5b-96c8-b15eca5574af None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:02.018143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-425ef8ab-405c-4c5b-96c8-b15eca5574af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.113029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1263/2545] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:02.113768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1283/2546] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:01 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:02.125367 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ef8ab-405c-4c5b-96c8-b15eca5574af None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:02.127326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-425ef8ab-405c-4c5b-96c8-b15eca5574af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.132613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeafa7-c20d-48fc-a6d6-858eeadde347 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:02.134096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8eeeafa7-c20d-48fc-a6d6-858eeadde347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.204289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ef8ab-405c-4c5b-96c8-b15eca5574af None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:02.205054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeafa7-c20d-48fc-a6d6-858eeadde347 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:02.208149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54330d92-baf4-4af4-aa88-277e716ff9ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:02.208931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54330d92-baf4-4af4-aa88-277e716ff9ac service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:02.209221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54330d92-baf4-4af4-aa88-277e716ff9ac service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:02.209340 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c0cf6-d354-40d1-a0ab-538c93a2913f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:02.209496 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54330d92-baf4-4af4-aa88-277e716ff9ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:02.209922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c0cf6-d354-40d1-a0ab-538c93a2913f service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:02.210205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c0cf6-d354-40d1-a0ab-538c93a2913f service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:02.210479 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c0cf6-d354-40d1-a0ab-538c93a2913f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:02.218581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:02.221549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c0cf6-d354-40d1-a0ab-538c93a2913f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:02.222750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54330d92-baf4-4af4-aa88-277e716ff9ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:02.223815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c0cf6-d354-40d1-a0ab-538c93a2913f service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:02.225014 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54330d92-baf4-4af4-aa88-277e716ff9ac service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:02.236181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1284/2547] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:02 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:02.237935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1264/2548] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:02.407067 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c0cf6-d354-40d1-a0ab-538c93a2913f service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:02.408647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9c9c0cf6-d354-40d1-a0ab-538c93a2913f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.426290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c0cf6-d354-40d1-a0ab-538c93a2913f service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:02.432897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:02.433891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:02.434245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:02.434613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] PATH_INFO: `/v3/users/9024325cab0a4109819d440713e9d8d5` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:02.436824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9024325cab0a4109819d440713e9d8d5)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:02.439456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:02.496334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9024325cab0a4109819d440713e9d8d5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.509903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9024325cab0a4109819d440713e9d8d5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.511223 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9024325cab0a4109819d440713e9d8d5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.511876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9024325cab0a4109819d440713e9d8d5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.524112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9024325cab0a4109819d440713e9d8d5'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.540349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1285/2549] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:02 2022] DELETE /identity/v3/users/9024325cab0a4109819d440713e9d8d5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:02.546180 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9061d-c9c0-4bd9-a245-eb83a4d19052 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:02.546859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9061d-c9c0-4bd9-a245-eb83a4d19052 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:02.546859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9061d-c9c0-4bd9-a245-eb83a4d19052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:02.547190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:02.548708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.574383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-bce9061d-c9c0-4bd9-a245-eb83a4d19052 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:02.575911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-bce9061d-c9c0-4bd9-a245-eb83a4d19052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.630511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ccb4-5abe-417f-90b2-0910cbc6d40e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:02.636299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:02.637216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:02.637499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:02.637775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] PATH_INFO: `/v3/projects/ac4f723b896943248837d999142f413b` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:02.639009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4f723b896943248837d999142f413b)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:02.640984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:02.651010 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1265/2550] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:02.661769 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9061d-c9c0-4bd9-a245-eb83a4d19052 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:02.663306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-bce9061d-c9c0-4bd9-a245-eb83a4d19052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.697300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:02.705846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4f723b896943248837d999142f413b', 'request_id': 'req-882a43db-d678-4b2e-8891-9dc62a3926ee'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.718127 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4f723b896943248837d999142f413b', 'request_id': 'req-882a43db-d678-4b2e-8891-9dc62a3926ee'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.728026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4f723b896943248837d999142f413b', 'request_id': 'req-882a43db-d678-4b2e-8891-9dc62a3926ee'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.729953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9061d-c9c0-4bd9-a245-eb83a4d19052 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:02.735669 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:02.741859 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] The token cache is being invalidate because project ac4f723b896943248837d999142f413b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:02.742260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.743170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-882a43db-d678-4b2e-8891-9dc62a3926ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:02.748624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1286/2551] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:02 2022] DELETE /identity/v3/projects/ac4f723b896943248837d999142f413b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:02.754729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506e7b3-9152-4af9-976c-237c64f6bbe0 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:02.755005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506e7b3-9152-4af9-976c-237c64f6bbe0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:02.755294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506e7b3-9152-4af9-976c-237c64f6bbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:02.794069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b506e7b3-9152-4af9-976c-237c64f6bbe0 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:02.795514 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b506e7b3-9152-4af9-976c-237c64f6bbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.800077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2504f-be96-4b18-ae37-272474144e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:02.800780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2504f-be96-4b18-ae37-272474144e0a service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:02.801056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2504f-be96-4b18-ae37-272474144e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:02.801333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2504f-be96-4b18-ae37-272474144e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:02.804723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-84e2504f-be96-4b18-ae37-272474144e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.898427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2504f-be96-4b18-ae37-272474144e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:02.899813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2504f-be96-4b18-ae37-272474144e0a service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:02.917101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1287/2552] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:02.917315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1266/2553] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:02.920819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada7538-bc7d-430c-bc6f-fe9a7843a00c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:02.929094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada7538-bc7d-430c-bc6f-fe9a7843a00c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:02.929526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada7538-bc7d-430c-bc6f-fe9a7843a00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:02.952422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-dada7538-bc7d-430c-bc6f-fe9a7843a00c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:02.954007 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-dada7538-bc7d-430c-bc6f-fe9a7843a00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:02.965659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1288/2554] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:02.970110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2504f-be96-4b18-ae37-272474144e0a service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:02.990838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2504f-be96-4b18-ae37-272474144e0a service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.006484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc27e1-c114-4ed3-85bf-dbaf9f66f356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.007640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc27e1-c114-4ed3-85bf-dbaf9f66f356 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.008034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc27e1-c114-4ed3-85bf-dbaf9f66f356 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.008424 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc27e1-c114-4ed3-85bf-dbaf9f66f356 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.009362 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc27e1-c114-4ed3-85bf-dbaf9f66f356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:03.009947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc27e1-c114-4ed3-85bf-dbaf9f66f356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.012701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc27e1-c114-4ed3-85bf-dbaf9f66f356 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.040227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1267/2555] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:03.042011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada7538-bc7d-430c-bc6f-fe9a7843a00c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.053226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada7538-bc7d-430c-bc6f-fe9a7843a00c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.059294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c9909-947a-4e57-bcc9-d1bd174d11fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.060094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c9909-947a-4e57-bcc9-d1bd174d11fd admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.060620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c9909-947a-4e57-bcc9-d1bd174d11fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.061000 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c9909-947a-4e57-bcc9-d1bd174d11fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.061810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c9909-947a-4e57-bcc9-d1bd174d11fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.063843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c9909-947a-4e57-bcc9-d1bd174d11fd admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.103123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1289/2556] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:03 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:03.108162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc27e1-c114-4ed3-85bf-dbaf9f66f356 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.119976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc27e1-c114-4ed3-85bf-dbaf9f66f356 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.233529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ae9f6-3ae8-41f4-bcc1-c0a77a73a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.235311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ae9f6-3ae8-41f4-bcc1-c0a77a73a043 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.235712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ae9f6-3ae8-41f4-bcc1-c0a77a73a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.236102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ae9f6-3ae8-41f4-bcc1-c0a77a73a043 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.236935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ae9f6-3ae8-41f4-bcc1-c0a77a73a043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.239324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ae9f6-3ae8-41f4-bcc1-c0a77a73a043 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.283197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1268/2557] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:03 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:03.285976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c9909-947a-4e57-bcc9-d1bd174d11fd admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.299452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c9909-947a-4e57-bcc9-d1bd174d11fd admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.305895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78aaf1b-8a01-47fa-b71e-e5e5e5adfdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.306963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aaf1b-8a01-47fa-b71e-e5e5e5adfdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.307323 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aaf1b-8a01-47fa-b71e-e5e5e5adfdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.307608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78aaf1b-8a01-47fa-b71e-e5e5e5adfdc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.308356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78aaf1b-8a01-47fa-b71e-e5e5e5adfdc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.310064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78aaf1b-8a01-47fa-b71e-e5e5e5adfdc2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.325536 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1290/2558] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:03.329388 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ae9f6-3ae8-41f4-bcc1-c0a77a73a043 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.342332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ae9f6-3ae8-41f4-bcc1-c0a77a73a043 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.359435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f1a4-86da-4bfd-a972-8317299f07d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.360738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f1a4-86da-4bfd-a972-8317299f07d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.361034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f1a4-86da-4bfd-a972-8317299f07d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.361317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663f1a4-86da-4bfd-a972-8317299f07d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.362074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663f1a4-86da-4bfd-a972-8317299f07d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.363703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663f1a4-86da-4bfd-a972-8317299f07d4 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.366723 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:03.397073 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1269/2559] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:03.400258 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78aaf1b-8a01-47fa-b71e-e5e5e5adfdc2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.417326 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f1a4-86da-4bfd-a972-8317299f07d4 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.419111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78aaf1b-8a01-47fa-b71e-e5e5e5adfdc2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.419628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0663f1a4-86da-4bfd-a972-8317299f07d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:03.426507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0068ab-cf46-46b8-82a4-18350284459e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.428204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0068ab-cf46-46b8-82a4-18350284459e admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.429535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0068ab-cf46-46b8-82a4-18350284459e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.430349 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0068ab-cf46-46b8-82a4-18350284459e admin admin] PATH_INFO: `/v3/projects/d5d1c68d6bfd4f1c9d2e57621e9859d3/users/6d2aa343872c411b8e9675188e641e38/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.459200 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663f1a4-86da-4bfd-a972-8317299f07d4 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.461071 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0068ab-cf46-46b8-82a4-18350284459e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d1c68d6bfd4f1c9d2e57621e9859d3, user_id=6d2aa343872c411b8e9675188e641e38, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.463884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0068ab-cf46-46b8-82a4-18350284459e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.469482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.471309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.471765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.472051 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] PATH_INFO: `/v3/projects/03f4862035d6419e84399d63bb8ff59a` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.473883 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03f4862035d6419e84399d63bb8ff59a)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.475782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.480471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1291/2560] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:03 2022] PUT /identity/v3/projects/d5d1c68d6bfd4f1c9d2e57621e9859d3/users/6d2aa343872c411b8e9675188e641e38/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:03.492141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f534b0-52ba-4206-a145-c5a59d116712 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.492726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f534b0-52ba-4206-a145-c5a59d116712 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.493052 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f534b0-52ba-4206-a145-c5a59d116712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.525268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-b2f534b0-52ba-4206-a145-c5a59d116712 None None] MFA Rules not processed for user `6d2aa343872c411b8e9675188e641e38`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:03.526497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b2f534b0-52ba-4206-a145-c5a59d116712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:03.531121 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03f4862035d6419e84399d63bb8ff59a', 'request_id': 'req-159feee9-5b6c-4592-bca5-ebe5df65f7e9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.540446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03f4862035d6419e84399d63bb8ff59a', 'request_id': 'req-159feee9-5b6c-4592-bca5-ebe5df65f7e9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.553728 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03f4862035d6419e84399d63bb8ff59a', 'request_id': 'req-159feee9-5b6c-4592-bca5-ebe5df65f7e9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.574482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] The token cache is being invalidate because project 03f4862035d6419e84399d63bb8ff59a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:03.574879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.575668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.577614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1270/2561] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:03 2022] DELETE /identity/v3/projects/03f4862035d6419e84399d63bb8ff59a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:03.581192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.582718 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:03.605306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1292/2562] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:03.609965 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f534b0-52ba-4206-a145-c5a59d116712 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.611391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b2f534b0-52ba-4206-a145-c5a59d116712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:03.641020 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:03.663960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159feee9-5b6c-4592-bca5-ebe5df65f7e9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.669309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.670199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.670537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.670885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] PATH_INFO: `/v3/users/7aaf0e53c6a744d7a48f5ecce890f6f1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.672610 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aaf0e53c6a744d7a48f5ecce890f6f1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.674595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.686427 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f534b0-52ba-4206-a145-c5a59d116712 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.691146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c8484-3620-4264-985e-b4d43c986d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.691813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c8484-3620-4264-985e-b4d43c986d04 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.692096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c8484-3620-4264-985e-b4d43c986d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.692429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c8484-3620-4264-985e-b4d43c986d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.693162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c8484-3620-4264-985e-b4d43c986d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.695209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c8484-3620-4264-985e-b4d43c986d04 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.733801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1293/2563] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:03 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:03.737713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c8484-3620-4264-985e-b4d43c986d04 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.738330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aaf0e53c6a744d7a48f5ecce890f6f1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.754541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aaf0e53c6a744d7a48f5ecce890f6f1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.763841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aaf0e53c6a744d7a48f5ecce890f6f1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.764912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aaf0e53c6a744d7a48f5ecce890f6f1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.773220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aaf0e53c6a744d7a48f5ecce890f6f1'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:03.776306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:03.783246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1271/2564] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:03 2022] DELETE /identity/v3/users/7aaf0e53c6a744d7a48f5ecce890f6f1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:03.839208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c8484-3620-4264-985e-b4d43c986d04 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.918519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cff0b7-7814-478c-882a-49a2e14bdee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.919252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cff0b7-7814-478c-882a-49a2e14bdee5 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.919601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cff0b7-7814-478c-882a-49a2e14bdee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.919806 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cff0b7-7814-478c-882a-49a2e14bdee5 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.920503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cff0b7-7814-478c-882a-49a2e14bdee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.922537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cff0b7-7814-478c-882a-49a2e14bdee5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:03.952624 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1294/2565] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:03 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:03.955192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:03.956694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:03.979099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6634-efc3-4177-aa18-17649df2d907 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:03.990633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec28b78-9091-4fb3-9c7b-550eb2f6fdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:03.991620 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec28b78-9091-4fb3-9c7b-550eb2f6fdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:03.991911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec28b78-9091-4fb3-9c7b-550eb2f6fdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:03.992210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec28b78-9091-4fb3-9c7b-550eb2f6fdb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:03.993032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec28b78-9091-4fb3-9c7b-550eb2f6fdb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:03.996213 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec28b78-9091-4fb3-9c7b-550eb2f6fdb3 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.007857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1272/2566] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:04.012844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cff0b7-7814-478c-882a-49a2e14bdee5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.023312 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cff0b7-7814-478c-882a-49a2e14bdee5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.029276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70beb93-9392-4d42-972b-0ebef98a7136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.030181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70beb93-9392-4d42-972b-0ebef98a7136 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.030465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70beb93-9392-4d42-972b-0ebef98a7136 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.030741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70beb93-9392-4d42-972b-0ebef98a7136 admin admin] PATH_INFO: `/v3/projects/b1fc4bd7799c4e7183046e32186c3dcd/users/10b134a35aa54b5ab471bb1fdd1846f6/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.045922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70beb93-9392-4d42-972b-0ebef98a7136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1fc4bd7799c4e7183046e32186c3dcd, user_id=10b134a35aa54b5ab471bb1fdd1846f6, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.048811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70beb93-9392-4d42-972b-0ebef98a7136 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.064007 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1295/2567] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:04 2022] PUT /identity/v3/projects/b1fc4bd7799c4e7183046e32186c3dcd/users/10b134a35aa54b5ab471bb1fdd1846f6/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:04.068158 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec28b78-9091-4fb3-9c7b-550eb2f6fdb3 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.080224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec28b78-9091-4fb3-9c7b-550eb2f6fdb3 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.129006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70beb93-9392-4d42-972b-0ebef98a7136 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.131477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b70beb93-9392-4d42-972b-0ebef98a7136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.154655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc0fab-0848-4a9a-bc7b-ca1f54c2043d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.155427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc0fab-0848-4a9a-bc7b-ca1f54c2043d admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.155783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc0fab-0848-4a9a-bc7b-ca1f54c2043d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.156178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc0fab-0848-4a9a-bc7b-ca1f54c2043d admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.158919 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc0fab-0848-4a9a-bc7b-ca1f54c2043d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.160545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc0fab-0848-4a9a-bc7b-ca1f54c2043d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.171490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:04.172795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1273/2568] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:56:04.176908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc0fab-0848-4a9a-bc7b-ca1f54c2043d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.188469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc0fab-0848-4a9a-bc7b-ca1f54c2043d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.193663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6d991-f452-45ff-aebc-e7da94216b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.194554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6d991-f452-45ff-aebc-e7da94216b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.194840 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6d991-f452-45ff-aebc-e7da94216b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.195179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6d991-f452-45ff-aebc-e7da94216b7c admin admin] PATH_INFO: `/v3/projects/b1fc4bd7799c4e7183046e32186c3dcd/users/10b134a35aa54b5ab471bb1fdd1846f6/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.197247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6d991-f452-45ff-aebc-e7da94216b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1fc4bd7799c4e7183046e32186c3dcd, user_id=10b134a35aa54b5ab471bb1fdd1846f6, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.200691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6d991-f452-45ff-aebc-e7da94216b7c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.212951 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1274/2569] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:04 2022] PUT /identity/v3/projects/b1fc4bd7799c4e7183046e32186c3dcd/users/10b134a35aa54b5ab471bb1fdd1846f6/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:04.218996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70beb93-9392-4d42-972b-0ebef98a7136 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.223108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74158b-bb8f-4358-b229-5021eb6ac2e0 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.223398 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74158b-bb8f-4358-b229-5021eb6ac2e0 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.223681 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74158b-bb8f-4358-b229-5021eb6ac2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.251976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-9c74158b-bb8f-4358-b229-5021eb6ac2e0 None None] MFA Rules not processed for user `10b134a35aa54b5ab471bb1fdd1846f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:04.253244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9c74158b-bb8f-4358-b229-5021eb6ac2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.280330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.281247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.281531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.281810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] PATH_INFO: `/v3/projects/ffdc945caf1f4526a7244f733d54234c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.283057 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffdc945caf1f4526a7244f733d54234c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.285062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.323611 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1275/2570] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:04.332185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffdc945caf1f4526a7244f733d54234c', 'request_id': 'req-0124970b-36a1-4ade-af16-1e075a7827fe'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.335000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74158b-bb8f-4358-b229-5021eb6ac2e0 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.336348 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9c74158b-bb8f-4358-b229-5021eb6ac2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.340266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffdc945caf1f4526a7244f733d54234c', 'request_id': 'req-0124970b-36a1-4ade-af16-1e075a7827fe'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.346786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffdc945caf1f4526a7244f733d54234c', 'request_id': 'req-0124970b-36a1-4ade-af16-1e075a7827fe'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.359982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] The token cache is being invalidate because project ffdc945caf1f4526a7244f733d54234c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:04.360717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.361007 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0124970b-36a1-4ade-af16-1e075a7827fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.362894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1296/2571] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:04 2022] DELETE /identity/v3/projects/ffdc945caf1f4526a7244f733d54234c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:04.367174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d8a5c-88d5-45c2-9755-64632d42442e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.367496 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d8a5c-88d5-45c2-9755-64632d42442e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.367781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d8a5c-88d5-45c2-9755-64632d42442e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.387418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f79d8a5c-88d5-45c2-9755-64632d42442e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:04.388650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f79d8a5c-88d5-45c2-9755-64632d42442e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.406680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74158b-bb8f-4358-b229-5021eb6ac2e0 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.416306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:04.453853 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:04.464168 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.465217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.465531 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.465777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.468011 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.471854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1297/2572] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:04.479439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8895c1-f6dd-4387-a0f1-0411c225e28e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.479727 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8895c1-f6dd-4387-a0f1-0411c225e28e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.480026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8895c1-f6dd-4387-a0f1-0411c225e28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.502512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2c8895c1-f6dd-4387-a0f1-0411c225e28e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:04.503732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2c8895c1-f6dd-4387-a0f1-0411c225e28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.520568 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1298/2573] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:04.521557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8895c1-f6dd-4387-a0f1-0411c225e28e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.531396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8895c1-f6dd-4387-a0f1-0411c225e28e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.537827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e2a9-d031-468b-b918-50ca832f4bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.538442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e2a9-d031-468b-b918-50ca832f4bcc admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.538726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e2a9-d031-468b-b918-50ca832f4bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.539814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137e2a9-d031-468b-b918-50ca832f4bcc admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.540562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e2a9-d031-468b-b918-50ca832f4bcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:04.540900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e2a9-d031-468b-b918-50ca832f4bcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.543747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137e2a9-d031-468b-b918-50ca832f4bcc admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.559761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1299/2574] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:04.561572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e2a9-d031-468b-b918-50ca832f4bcc admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.563742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.565595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.571961 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137e2a9-d031-468b-b918-50ca832f4bcc admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.576064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1276/2575] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:04.577831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef8e4-d340-4c09-b28c-b144ee061ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.578640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef8e4-d340-4c09-b28c-b144ee061ead admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.578931 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef8e4-d340-4c09-b28c-b144ee061ead admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.579208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef8e4-d340-4c09-b28c-b144ee061ead admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.579902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bef8e4-d340-4c09-b28c-b144ee061ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.581845 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bef8e4-d340-4c09-b28c-b144ee061ead admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.615103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.617036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.621412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1300/2576] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:04 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:04.625466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef8e4-d340-4c09-b28c-b144ee061ead admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.631904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb0bfd-80a4-4247-af97-3661dba3ffed service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.700266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef8e4-d340-4c09-b28c-b144ee061ead admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.703095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.704551 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.704981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.705353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] PATH_INFO: `/v3/users/6d2aa343872c411b8e9675188e641e38` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.706833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2aa343872c411b8e9675188e641e38)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.709092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.714383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38171c-a07a-4fdb-af8f-f30424698b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.715172 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38171c-a07a-4fdb-af8f-f30424698b5f admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.715462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38171c-a07a-4fdb-af8f-f30424698b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.715866 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38171c-a07a-4fdb-af8f-f30424698b5f admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.716590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38171c-a07a-4fdb-af8f-f30424698b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.718696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38171c-a07a-4fdb-af8f-f30424698b5f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.746688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1301/2577] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:04 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:04.751605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38171c-a07a-4fdb-af8f-f30424698b5f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.765922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38171c-a07a-4fdb-af8f-f30424698b5f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.772746 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc3a3-dc09-4bc2-be21-fe682c0b456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.774200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc3a3-dc09-4bc2-be21-fe682c0b456c admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.774700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc3a3-dc09-4bc2-be21-fe682c0b456c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.775495 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764cc3a3-dc09-4bc2-be21-fe682c0b456c admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.776844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc3a3-dc09-4bc2-be21-fe682c0b456c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.780146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764cc3a3-dc09-4bc2-be21-fe682c0b456c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.786060 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2aa343872c411b8e9675188e641e38'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.796629 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1302/2578] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:04.798074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc3a3-dc09-4bc2-be21-fe682c0b456c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.800909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2aa343872c411b8e9675188e641e38'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.802689 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:04.809656 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764cc3a3-dc09-4bc2-be21-fe682c0b456c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.812684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2aa343872c411b8e9675188e641e38'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.815024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2aa343872c411b8e9675188e641e38'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.815290 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f081d2-3ae8-46c8-90b3-b6dcc8c0ca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.816040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f081d2-3ae8-46c8-90b3-b6dcc8c0ca77 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.816459 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f081d2-3ae8-46c8-90b3-b6dcc8c0ca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.816831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f081d2-3ae8-46c8-90b3-b6dcc8c0ca77 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.817633 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f081d2-3ae8-46c8-90b3-b6dcc8c0ca77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.819412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f081d2-3ae8-46c8-90b3-b6dcc8c0ca77 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.825538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-f256992d-7a4e-4a2e-a859-239c172a63bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2aa343872c411b8e9675188e641e38'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:04.831533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1303/2579] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:04.832212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1277/2580] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:04 2022] DELETE /identity/v3/users/6d2aa343872c411b8e9675188e641e38 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:04.835573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f081d2-3ae8-46c8-90b3-b6dcc8c0ca77 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.836498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a98685-de15-4747-b0ef-849eca348216 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.836813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a98685-de15-4747-b0ef-849eca348216 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.837090 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a98685-de15-4747-b0ef-849eca348216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.837324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-12f081d2-3ae8-46c8-90b3-b6dcc8c0ca77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.851245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f081d2-3ae8-46c8-90b3-b6dcc8c0ca77 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:04.855379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-66a98685-de15-4747-b0ef-849eca348216 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:04.856632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-66a98685-de15-4747-b0ef-849eca348216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.913526 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dea1a0-1c1c-4c24-8caf-362da52f19ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:04.914329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dea1a0-1c1c-4c24-8caf-362da52f19ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.914614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dea1a0-1c1c-4c24-8caf-362da52f19ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.914886 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dea1a0-1c1c-4c24-8caf-362da52f19ed admin admin] PATH_INFO: `/v3/projects/063089da5f7b4051930d1ee0c1603341/users/fb2897368fc44fa3b5a3442648d78005/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.917684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1278/2581] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:04.927464 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a98685-de15-4747-b0ef-849eca348216 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:04.928887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-66a98685-de15-4747-b0ef-849eca348216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.929869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dea1a0-1c1c-4c24-8caf-362da52f19ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=063089da5f7b4051930d1ee0c1603341, user_id=fb2897368fc44fa3b5a3442648d78005, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:04.933573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dea1a0-1c1c-4c24-8caf-362da52f19ed admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:04.948620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1304/2582] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:04 2022] PUT /identity/v3/projects/063089da5f7b4051930d1ee0c1603341/users/fb2897368fc44fa3b5a3442648d78005/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:04.960072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04681df-177a-4e45-9661-aa435579f833 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:04.960355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04681df-177a-4e45-9661-aa435579f833 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:04.960703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04681df-177a-4e45-9661-aa435579f833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:04.978958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:04.990308 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-f04681df-177a-4e45-9661-aa435579f833 None None] MFA Rules not processed for user `fb2897368fc44fa3b5a3442648d78005`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:04.991434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f04681df-177a-4e45-9661-aa435579f833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:04.992431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a98685-de15-4747-b0ef-849eca348216 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.053626 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efce27-4bfa-4918-bdf8-cb54af41c2dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.054312 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efce27-4bfa-4918-bdf8-cb54af41c2dc service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.057437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efce27-4bfa-4918-bdf8-cb54af41c2dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.057437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efce27-4bfa-4918-bdf8-cb54af41c2dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.078273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1305/2583] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:05.082959 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04681df-177a-4e45-9661-aa435579f833 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:05.121661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:05.135072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efce27-4bfa-4918-bdf8-cb54af41c2dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.137292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efce27-4bfa-4918-bdf8-cb54af41c2dc service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.150433 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1279/2584] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:05.152337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04681df-177a-4e45-9661-aa435579f833 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.158407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a78dc4-e186-4447-9da9-1df4359f19c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.158940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a78dc4-e186-4447-9da9-1df4359f19c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.159237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a78dc4-e186-4447-9da9-1df4359f19c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.159505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a78dc4-e186-4447-9da9-1df4359f19c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.160208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a78dc4-e186-4447-9da9-1df4359f19c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.162335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a78dc4-e186-4447-9da9-1df4359f19c5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.193967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1306/2585] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:05 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:05.198390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efce27-4bfa-4918-bdf8-cb54af41c2dc service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:05.211229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efce27-4bfa-4918-bdf8-cb54af41c2dc service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.288747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e15da4-c91f-49c6-9db0-2d1d3e41e25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.289498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e15da4-c91f-49c6-9db0-2d1d3e41e25f admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.289874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e15da4-c91f-49c6-9db0-2d1d3e41e25f admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.290216 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e15da4-c91f-49c6-9db0-2d1d3e41e25f admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.290985 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e15da4-c91f-49c6-9db0-2d1d3e41e25f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.293598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e15da4-c91f-49c6-9db0-2d1d3e41e25f admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.318436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1280/2586] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:05 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:05.319543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a78dc4-e186-4447-9da9-1df4359f19c5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:05.343445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a78dc4-e186-4447-9da9-1df4359f19c5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.348364 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a601-f191-40de-9708-1726c24ef9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.349142 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a601-f191-40de-9708-1726c24ef9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.349501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a601-f191-40de-9708-1726c24ef9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.349860 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a601-f191-40de-9708-1726c24ef9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.350706 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856a601-f191-40de-9708-1726c24ef9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.352441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856a601-f191-40de-9708-1726c24ef9c9 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.363392 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1307/2587] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:56:05.367731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e15da4-c91f-49c6-9db0-2d1d3e41e25f admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:05.378941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e15da4-c91f-49c6-9db0-2d1d3e41e25f admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.384254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db00efc-2a0e-4a16-b00b-554dcaf51c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.385782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db00efc-2a0e-4a16-b00b-554dcaf51c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.386088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db00efc-2a0e-4a16-b00b-554dcaf51c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.386456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db00efc-2a0e-4a16-b00b-554dcaf51c44 admin admin] PATH_INFO: `/v3/projects/fd9160a84d1c4f6998dea326f089c979/users/6e8e47a4ed964d429b58eedd55a581ef/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.403287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db00efc-2a0e-4a16-b00b-554dcaf51c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9160a84d1c4f6998dea326f089c979, user_id=6e8e47a4ed964d429b58eedd55a581ef, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.406402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db00efc-2a0e-4a16-b00b-554dcaf51c44 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.422519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1281/2588] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:05 2022] PUT /identity/v3/projects/fd9160a84d1c4f6998dea326f089c979/users/6e8e47a4ed964d429b58eedd55a581ef/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:05.426771 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a601-f191-40de-9708-1726c24ef9c9 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:05.438687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a601-f191-40de-9708-1726c24ef9c9 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.465054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db00efc-2a0e-4a16-b00b-554dcaf51c44 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:05.466875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3db00efc-2a0e-4a16-b00b-554dcaf51c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:05.514777 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb053cf-e2b3-4710-9929-517aeb6e9f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.515458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb053cf-e2b3-4710-9929-517aeb6e9f31 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.515747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb053cf-e2b3-4710-9929-517aeb6e9f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.516023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb053cf-e2b3-4710-9929-517aeb6e9f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.516798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb053cf-e2b3-4710-9929-517aeb6e9f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.518697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb053cf-e2b3-4710-9929-517aeb6e9f31 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.533712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1308/2589] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:05.535104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb053cf-e2b3-4710-9929-517aeb6e9f31 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:05.545914 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb053cf-e2b3-4710-9929-517aeb6e9f31 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.550713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a093e-d508-48d9-83ad-fc022b4426ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.551602 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a093e-d508-48d9-83ad-fc022b4426ad admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.551888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a093e-d508-48d9-83ad-fc022b4426ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.552294 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a093e-d508-48d9-83ad-fc022b4426ad admin admin] PATH_INFO: `/v3/projects/fd9160a84d1c4f6998dea326f089c979/users/6e8e47a4ed964d429b58eedd55a581ef/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.554315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a093e-d508-48d9-83ad-fc022b4426ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9160a84d1c4f6998dea326f089c979, user_id=6e8e47a4ed964d429b58eedd55a581ef, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.557608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a093e-d508-48d9-83ad-fc022b4426ad admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.561025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:05.570669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db00efc-2a0e-4a16-b00b-554dcaf51c44 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.570875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1309/2590] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:05 2022] PUT /identity/v3/projects/fd9160a84d1c4f6998dea326f089c979/users/6e8e47a4ed964d429b58eedd55a581ef/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:05.580989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d335c-9cec-496e-b525-e0d1b154632d None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.581819 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d335c-9cec-496e-b525-e0d1b154632d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.582188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d335c-9cec-496e-b525-e0d1b154632d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.611386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-712d335c-9cec-496e-b525-e0d1b154632d None None] MFA Rules not processed for user `6e8e47a4ed964d429b58eedd55a581ef`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:05.612614 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-712d335c-9cec-496e-b525-e0d1b154632d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:05.636755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.637651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.637935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.638214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] PATH_INFO: `/v3/projects/d5d1c68d6bfd4f1c9d2e57621e9859d3` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.639436 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d1c68d6bfd4f1c9d2e57621e9859d3)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.641421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.643926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:05.691035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1310/2591] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:05.693166 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d1c68d6bfd4f1c9d2e57621e9859d3', 'request_id': 'req-07833c57-a59a-4c7e-b71a-6dad6252773b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.700030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d1c68d6bfd4f1c9d2e57621e9859d3', 'request_id': 'req-07833c57-a59a-4c7e-b71a-6dad6252773b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.700204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d335c-9cec-496e-b525-e0d1b154632d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:05.701585 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-712d335c-9cec-496e-b525-e0d1b154632d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:05.708239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d1c68d6bfd4f1c9d2e57621e9859d3', 'request_id': 'req-07833c57-a59a-4c7e-b71a-6dad6252773b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.719809 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] The token cache is being invalidate because project d5d1c68d6bfd4f1c9d2e57621e9859d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:05.720098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.720771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.722316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1282/2592] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:05 2022] DELETE /identity/v3/projects/d5d1c68d6bfd4f1c9d2e57621e9859d3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:05.726087 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:05.727396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:05.763840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d335c-9cec-496e-b525-e0d1b154632d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.797040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07833c57-a59a-4c7e-b71a-6dad6252773b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:05.802407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.803287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.803570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.803850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] PATH_INFO: `/v3/users/10b134a35aa54b5ab471bb1fdd1846f6` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.805171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b134a35aa54b5ab471bb1fdd1846f6)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.808237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.817991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59189c7a-ad16-44ec-9e53-ad4c90c611c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:05.818645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59189c7a-ad16-44ec-9e53-ad4c90c611c1 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:05.818953 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59189c7a-ad16-44ec-9e53-ad4c90c611c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:05.819256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59189c7a-ad16-44ec-9e53-ad4c90c611c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:05.821281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-59189c7a-ad16-44ec-9e53-ad4c90c611c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:05.874576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b134a35aa54b5ab471bb1fdd1846f6'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.888086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:05.891122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b134a35aa54b5ab471bb1fdd1846f6'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.899405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59189c7a-ad16-44ec-9e53-ad4c90c611c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:05.900869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b134a35aa54b5ab471bb1fdd1846f6'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.901380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59189c7a-ad16-44ec-9e53-ad4c90c611c1 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:05.901696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b134a35aa54b5ab471bb1fdd1846f6'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.909512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b134a35aa54b5ab471bb1fdd1846f6'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:05.911991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1311/2593] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:05.915534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1283/2594] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:05 2022] DELETE /identity/v3/users/10b134a35aa54b5ab471bb1fdd1846f6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:06.253516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59189c7a-ad16-44ec-9e53-ad4c90c611c1 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:06.275494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59189c7a-ad16-44ec-9e53-ad4c90c611c1 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:06.373270 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:06.374458 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.374652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.375576 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] PATH_INFO: `/v3/projects/b1fc4bd7799c4e7183046e32186c3dcd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.377042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1fc4bd7799c4e7183046e32186c3dcd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:06.378942 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:06.419685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:06.435398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1fc4bd7799c4e7183046e32186c3dcd', 'request_id': 'req-fbee8dde-e079-454a-93d9-954b3398b414'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.435493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:06.441701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1fc4bd7799c4e7183046e32186c3dcd', 'request_id': 'req-fbee8dde-e079-454a-93d9-954b3398b414'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.451149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1fc4bd7799c4e7183046e32186c3dcd', 'request_id': 'req-fbee8dde-e079-454a-93d9-954b3398b414'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.457524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b03fb-316b-44cf-a09e-a29193fa4bcf admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:06.462788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] The token cache is being invalidate because project b1fc4bd7799c4e7183046e32186c3dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:06.463232 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.464171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.466241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:06.466601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1312/2595] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:06 2022] DELETE /identity/v3/projects/b1fc4bd7799c4e7183046e32186c3dcd => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:06.467816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:06.471840 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:06.532108 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee8dde-e079-454a-93d9-954b3398b414 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:06.534390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:06.535467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.535821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.536176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] PATH_INFO: `/v3/users/fb2897368fc44fa3b5a3442648d78005` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.537027 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe01ac6-0104-4db3-8066-8d6137933aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:06.537632 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb2897368fc44fa3b5a3442648d78005)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:06.537989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe01ac6-0104-4db3-8066-8d6137933aba service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.538522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe01ac6-0104-4db3-8066-8d6137933aba service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.538945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe01ac6-0104-4db3-8066-8d6137933aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.540570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:06.541176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-efe01ac6-0104-4db3-8066-8d6137933aba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:06.554149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe01ac6-0104-4db3-8066-8d6137933aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:06.556061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe01ac6-0104-4db3-8066-8d6137933aba service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:06.567500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1313/2596] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:06.570144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dcddb-3476-4352-acf4-f8042d8314ae None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.570487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dcddb-3476-4352-acf4-f8042d8314ae None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.570802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dcddb-3476-4352-acf4-f8042d8314ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.596399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-305dcddb-3476-4352-acf4-f8042d8314ae None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:06.597765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-305dcddb-3476-4352-acf4-f8042d8314ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:06.608195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb2897368fc44fa3b5a3442648d78005'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.616165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1314/2597] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:06.624558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61185cac-75b2-4ab9-aed0-044954352aa0 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.625083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61185cac-75b2-4ab9-aed0-044954352aa0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.625378 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61185cac-75b2-4ab9-aed0-044954352aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.632936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb2897368fc44fa3b5a3442648d78005'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.646422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb2897368fc44fa3b5a3442648d78005'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.647218 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb2897368fc44fa3b5a3442648d78005'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.652229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-61185cac-75b2-4ab9-aed0-044954352aa0 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:06.654765 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-61185cac-75b2-4ab9-aed0-044954352aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:06.655848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7b902eb4-8bf3-4676-b0a4-5050f1649908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb2897368fc44fa3b5a3442648d78005'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:06.665866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1284/2598] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:06 2022] DELETE /identity/v3/users/fb2897368fc44fa3b5a3442648d78005 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:06.668471 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1315/2599] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:06.672085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36660d71-7c65-4cd8-b04e-a81042227dec None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.672085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36660d71-7c65-4cd8-b04e-a81042227dec None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.672085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36660d71-7c65-4cd8-b04e-a81042227dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.673647 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61185cac-75b2-4ab9-aed0-044954352aa0 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:06.684908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61185cac-75b2-4ab9-aed0-044954352aa0 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:06.689797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:06.702912 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-36660d71-7c65-4cd8-b04e-a81042227dec None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:06.704476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-36660d71-7c65-4cd8-b04e-a81042227dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:06.787709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1285/2600] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:06.797119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f4bc4-f8a4-413d-a57d-29e70072d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:06.798008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f4bc4-f8a4-413d-a57d-29e70072d75f admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.798663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f4bc4-f8a4-413d-a57d-29e70072d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.799042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f4bc4-f8a4-413d-a57d-29e70072d75f admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.799454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36660d71-7c65-4cd8-b04e-a81042227dec None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:06.799974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f4bc4-f8a4-413d-a57d-29e70072d75f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:06.800384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f4bc4-f8a4-413d-a57d-29e70072d75f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:06.801676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-36660d71-7c65-4cd8-b04e-a81042227dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:06.802282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f4bc4-f8a4-413d-a57d-29e70072d75f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:06.816971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1316/2601] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:06.821016 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f4bc4-f8a4-413d-a57d-29e70072d75f admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:06.834523 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f4bc4-f8a4-413d-a57d-29e70072d75f admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:06.841175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ef7df-4ded-4613-90b3-3cae175268e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:06.841921 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ef7df-4ded-4613-90b3-3cae175268e6 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.842632 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ef7df-4ded-4613-90b3-3cae175268e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.842991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ef7df-4ded-4613-90b3-3cae175268e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.844005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ef7df-4ded-4613-90b3-3cae175268e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:06.846651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ef7df-4ded-4613-90b3-3cae175268e6 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:06.849253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:06.883103 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1317/2602] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:06 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:06.883244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36660d71-7c65-4cd8-b04e-a81042227dec None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:06.887838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ef7df-4ded-4613-90b3-3cae175268e6 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:06.953449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7ff04-1539-451b-93d2-970f0a6c9a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:06.954024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7ff04-1539-451b-93d2-970f0a6c9a7a service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.954305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7ff04-1539-451b-93d2-970f0a6c9a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.954582 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7ff04-1539-451b-93d2-970f0a6c9a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.973839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ef7df-4ded-4613-90b3-3cae175268e6 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:06.978610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67355a2b-1679-4d83-a12c-ca894f954ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:06.979778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67355a2b-1679-4d83-a12c-ca894f954ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:06.980205 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67355a2b-1679-4d83-a12c-ca894f954ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:06.980485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67355a2b-1679-4d83-a12c-ca894f954ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:06.981236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67355a2b-1679-4d83-a12c-ca894f954ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:06.983256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67355a2b-1679-4d83-a12c-ca894f954ca7 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:07.009497 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1318/2603] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:06 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:07.010906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67355a2b-1679-4d83-a12c-ca894f954ca7 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:07.022363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67355a2b-1679-4d83-a12c-ca894f954ca7 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:07.028059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f6807-7016-4e36-8ab5-ac956fc3ebaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:07.028800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f6807-7016-4e36-8ab5-ac956fc3ebaf admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:07.029169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f6807-7016-4e36-8ab5-ac956fc3ebaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:07.029443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f6807-7016-4e36-8ab5-ac956fc3ebaf admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:07.030280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f6807-7016-4e36-8ab5-ac956fc3ebaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:07.032178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f6807-7016-4e36-8ab5-ac956fc3ebaf admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:07.048026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1319/2604] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:07.048227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7ff04-1539-451b-93d2-970f0a6c9a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:07.048959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7ff04-1539-451b-93d2-970f0a6c9a7a service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:07.049185 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f6807-7016-4e36-8ab5-ac956fc3ebaf admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:07.062036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f6807-7016-4e36-8ab5-ac956fc3ebaf admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:07.072680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6483fc-5806-4557-9961-68ccf67bd3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:07.074207 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6483fc-5806-4557-9961-68ccf67bd3ef admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:07.074508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6483fc-5806-4557-9961-68ccf67bd3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:07.074903 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6483fc-5806-4557-9961-68ccf67bd3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:07.075700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6483fc-5806-4557-9961-68ccf67bd3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:07.079575 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6483fc-5806-4557-9961-68ccf67bd3ef admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:07.080702 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1286/2605] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:07.092493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1320/2606] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:07.096658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7ff04-1539-451b-93d2-970f0a6c9a7a service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:07.108854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7ff04-1539-451b-93d2-970f0a6c9a7a service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:07.116982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace77b9-eff4-4b21-9be9-ff8ca68f4d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:07.118035 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace77b9-eff4-4b21-9be9-ff8ca68f4d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:07.118394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace77b9-eff4-4b21-9be9-ff8ca68f4d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:07.118784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace77b9-eff4-4b21-9be9-ff8ca68f4d50 admin admin] PATH_INFO: `/v3/projects/f838b0cec2514a80a23c3259149dfd75/users/487de32b2e264570a939ea6e8ab1cfdd/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:07.141334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace77b9-eff4-4b21-9be9-ff8ca68f4d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f838b0cec2514a80a23c3259149dfd75, user_id=487de32b2e264570a939ea6e8ab1cfdd, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:07.144456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace77b9-eff4-4b21-9be9-ff8ca68f4d50 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:07.166088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1287/2607] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:07 2022] PUT /identity/v3/projects/f838b0cec2514a80a23c3259149dfd75/users/487de32b2e264570a939ea6e8ab1cfdd/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:07.178157 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ad58c-0778-448d-b101-3f5d3ac153bf None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:07.178465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ad58c-0778-448d-b101-3f5d3ac153bf None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:07.178838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ad58c-0778-448d-b101-3f5d3ac153bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:07.214299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-477ad58c-0778-448d-b101-3f5d3ac153bf None None] MFA Rules not processed for user `487de32b2e264570a939ea6e8ab1cfdd`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:07.215559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-477ad58c-0778-448d-b101-3f5d3ac153bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:07.292982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:07.348716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1321/2608] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:07.358446 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace77b9-eff4-4b21-9be9-ff8ca68f4d50 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:07.360574 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0ace77b9-eff4-4b21-9be9-ff8ca68f4d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:07.449708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ad58c-0778-448d-b101-3f5d3ac153bf None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:07.451643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-477ad58c-0778-448d-b101-3f5d3ac153bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:07.476238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace77b9-eff4-4b21-9be9-ff8ca68f4d50 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:07.481002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f6bbf-426e-44be-af33-f554e64ef01a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:07.481669 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f6bbf-426e-44be-af33-f554e64ef01a service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:07.482009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f6bbf-426e-44be-af33-f554e64ef01a service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:07.482534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f6bbf-426e-44be-af33-f554e64ef01a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:07.493079 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f6bbf-426e-44be-af33-f554e64ef01a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:07.496089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f6bbf-426e-44be-af33-f554e64ef01a service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:07.508043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1288/2609] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:07.534102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f6bbf-426e-44be-af33-f554e64ef01a service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:07.556427 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:07.599652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ad58c-0778-448d-b101-3f5d3ac153bf None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:07.609780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:07.610781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:07.611065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:07.611341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] PATH_INFO: `/v3/projects/063089da5f7b4051930d1ee0c1603341` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:07.612714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=063089da5f7b4051930d1ee0c1603341)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:07.615251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:07.634018 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f6bbf-426e-44be-af33-f554e64ef01a service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:07.643327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:07.644850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:07.645135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:07.645462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] PATH_INFO: `/v3/users/487de32b2e264570a939ea6e8ab1cfdd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:07.650191 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=487de32b2e264570a939ea6e8ab1cfdd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:07.653259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:07.688551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063089da5f7b4051930d1ee0c1603341', 'request_id': 'req-c4bff12a-0550-489b-b29e-bdd664568f67'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.700383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063089da5f7b4051930d1ee0c1603341', 'request_id': 'req-c4bff12a-0550-489b-b29e-bdd664568f67'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.712199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063089da5f7b4051930d1ee0c1603341', 'request_id': 'req-c4bff12a-0550-489b-b29e-bdd664568f67'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.726828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] The token cache is being invalidate because project 063089da5f7b4051930d1ee0c1603341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:07.727125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.727979 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.732296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1322/2610] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:07 2022] DELETE /identity/v3/projects/063089da5f7b4051930d1ee0c1603341 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:07.733683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '487de32b2e264570a939ea6e8ab1cfdd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.735703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:07.739062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:07.762179 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '487de32b2e264570a939ea6e8ab1cfdd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.781315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '487de32b2e264570a939ea6e8ab1cfdd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.781963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '487de32b2e264570a939ea6e8ab1cfdd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.793605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-77ad2452-c1ab-4f59-8810-86a2be21bdfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '487de32b2e264570a939ea6e8ab1cfdd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:07.800874 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1289/2611] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:07 2022] DELETE /identity/v3/users/487de32b2e264570a939ea6e8ab1cfdd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:07.805553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2d7cf-8738-480a-b7cf-3fdd86048c23 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:07.806124 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2d7cf-8738-480a-b7cf-3fdd86048c23 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:07.806585 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2d7cf-8738-480a-b7cf-3fdd86048c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:07.828685 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-bdc2d7cf-8738-480a-b7cf-3fdd86048c23 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:07.830194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-bdc2d7cf-8738-480a-b7cf-3fdd86048c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:07.837736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bff12a-0550-489b-b29e-bdd664568f67 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:07.849462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:07.950117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1290/2612] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:07.959905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:07.964134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:07.964639 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:07.964892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2d7cf-8738-480a-b7cf-3fdd86048c23 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:07.965021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] PATH_INFO: `/v3/users/6e8e47a4ed964d429b58eedd55a581ef` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:07.966772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8e47a4ed964d429b58eedd55a581ef)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:07.966909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-bdc2d7cf-8738-480a-b7cf-3fdd86048c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:07.971857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:08.036955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:08.072055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2d7cf-8738-480a-b7cf-3fdd86048c23 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:08.073733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8e47a4ed964d429b58eedd55a581ef'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.087506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef841c95-0aa7-4821-954f-96270ee7bbcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:08.089638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef841c95-0aa7-4821-954f-96270ee7bbcc service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:08.096137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef841c95-0aa7-4821-954f-96270ee7bbcc service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:08.096425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef841c95-0aa7-4821-954f-96270ee7bbcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:08.114717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef841c95-0aa7-4821-954f-96270ee7bbcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:08.115672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8e47a4ed964d429b58eedd55a581ef'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.116350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef841c95-0aa7-4821-954f-96270ee7bbcc service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:08.116466 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8e47a4ed964d429b58eedd55a581ef'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.117201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8e47a4ed964d429b58eedd55a581ef'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.128149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-2324fb65-7f72-424c-a7a7-8dc80ff7f3e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8e47a4ed964d429b58eedd55a581ef'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.130056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1291/2613] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:08.169227 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1323/2614] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:07 2022] DELETE /identity/v3/users/6e8e47a4ed964d429b58eedd55a581ef => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:08.472602 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef841c95-0aa7-4821-954f-96270ee7bbcc service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:08.475878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ef841c95-0aa7-4821-954f-96270ee7bbcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:08.511608 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef841c95-0aa7-4821-954f-96270ee7bbcc service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:08.589515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:08.589515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:08.589515 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:08.589946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] PATH_INFO: `/v3/projects/f838b0cec2514a80a23c3259149dfd75` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:08.591063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f838b0cec2514a80a23c3259149dfd75)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:08.593042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:08.657643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f838b0cec2514a80a23c3259149dfd75', 'request_id': 'req-ce42ae2d-9011-4cea-aec8-10000820c58a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.667116 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f838b0cec2514a80a23c3259149dfd75', 'request_id': 'req-ce42ae2d-9011-4cea-aec8-10000820c58a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.675243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f838b0cec2514a80a23c3259149dfd75', 'request_id': 'req-ce42ae2d-9011-4cea-aec8-10000820c58a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.687028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] The token cache is being invalidate because project f838b0cec2514a80a23c3259149dfd75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:08.687310 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.687939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ce42ae2d-9011-4cea-aec8-10000820c58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.689720 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1292/2615] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:08 2022] DELETE /identity/v3/projects/f838b0cec2514a80a23c3259149dfd75 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:08.694341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62850b2e-6754-46c2-b694-bea61f9fccab None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:08.694894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62850b2e-6754-46c2-b694-bea61f9fccab None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:08.694994 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62850b2e-6754-46c2-b694-bea61f9fccab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:08.719967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-62850b2e-6754-46c2-b694-bea61f9fccab None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:08.721230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-62850b2e-6754-46c2-b694-bea61f9fccab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:08.791091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1324/2616] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:08.799978 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a3398-897b-4279-8afd-9fc7fb271b78 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:08.800275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a3398-897b-4279-8afd-9fc7fb271b78 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:08.800559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a3398-897b-4279-8afd-9fc7fb271b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:08.824857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-463a3398-897b-4279-8afd-9fc7fb271b78 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:08.824857 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-463a3398-897b-4279-8afd-9fc7fb271b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:08.826314 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62850b2e-6754-46c2-b694-bea61f9fccab None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:08.827915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-62850b2e-6754-46c2-b694-bea61f9fccab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:08.834105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1293/2617] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:08.839484 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a3398-897b-4279-8afd-9fc7fb271b78 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:08.850838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62850b2e-6754-46c2-b694-bea61f9fccab None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:08.851075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a3398-897b-4279-8afd-9fc7fb271b78 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:08.857807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170bb83-0135-4211-b399-6f6f4bd46047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:08.858696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170bb83-0135-4211-b399-6f6f4bd46047 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:08.859035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:08.859181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170bb83-0135-4211-b399-6f6f4bd46047 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:08.859504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170bb83-0135-4211-b399-6f6f4bd46047 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:08.860300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:08.860447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170bb83-0135-4211-b399-6f6f4bd46047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:08.860622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:08.860927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] PATH_INFO: `/v3/projects/fd9160a84d1c4f6998dea326f089c979` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:08.861049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170bb83-0135-4211-b399-6f6f4bd46047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:08.863061 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170bb83-0135-4211-b399-6f6f4bd46047 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:08.863388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9160a84d1c4f6998dea326f089c979)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:08.865742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:08.884684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1294/2618] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:56:08.891189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170bb83-0135-4211-b399-6f6f4bd46047 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:08.906350 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170bb83-0135-4211-b399-6f6f4bd46047 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:08.911206 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c819ec-cdf8-4dc2-b034-a754c84a40a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:08.911872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c819ec-cdf8-4dc2-b034-a754c84a40a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:08.912152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c819ec-cdf8-4dc2-b034-a754c84a40a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:08.912482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c819ec-cdf8-4dc2-b034-a754c84a40a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:08.913407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c819ec-cdf8-4dc2-b034-a754c84a40a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:08.915434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c819ec-cdf8-4dc2-b034-a754c84a40a4 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:08.937258 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:08.960279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9160a84d1c4f6998dea326f089c979', 'request_id': 'req-dd9d4316-d3f7-4c07-8403-2603926cb91f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.971905 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1295/2619] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:08 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:08.973004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c819ec-cdf8-4dc2-b034-a754c84a40a4 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:08.978191 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9160a84d1c4f6998dea326f089c979', 'request_id': 'req-dd9d4316-d3f7-4c07-8403-2603926cb91f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:08.988453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9160a84d1c4f6998dea326f089c979', 'request_id': 'req-dd9d4316-d3f7-4c07-8403-2603926cb91f'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:09.004317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] The token cache is being invalidate because project fd9160a84d1c4f6998dea326f089c979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:09.004954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:09.006046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-dd9d4316-d3f7-4c07-8403-2603926cb91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:09.008872 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1325/2620] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:08 2022] DELETE /identity/v3/projects/fd9160a84d1c4f6998dea326f089c979 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:09.010131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:09.014112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561029-93f7-4e02-aaf8-c896be70c791 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.014562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561029-93f7-4e02-aaf8-c896be70c791 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.015083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561029-93f7-4e02-aaf8-c896be70c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.044828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-53561029-93f7-4e02-aaf8-c896be70c791 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:09.046761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-53561029-93f7-4e02-aaf8-c896be70c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:09.055795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1326/2621] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:09.060299 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c819ec-cdf8-4dc2-b034-a754c84a40a4 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.064134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8da83c-3a1b-40bc-b320-733dc4df2b63 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.064491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8da83c-3a1b-40bc-b320-733dc4df2b63 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.064732 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8da83c-3a1b-40bc-b320-733dc4df2b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.067449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb1c97-c983-4ecc-8429-25b7d4ea9306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.068223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb1c97-c983-4ecc-8429-25b7d4ea9306 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.068508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb1c97-c983-4ecc-8429-25b7d4ea9306 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.068949 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb1c97-c983-4ecc-8429-25b7d4ea9306 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.069697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb1c97-c983-4ecc-8429-25b7d4ea9306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.071743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb1c97-c983-4ecc-8429-25b7d4ea9306 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.087251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3c8da83c-3a1b-40bc-b320-733dc4df2b63 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:09.090284 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3c8da83c-3a1b-40bc-b320-733dc4df2b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:09.098747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1296/2622] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:08 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:09.103000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb1c97-c983-4ecc-8429-25b7d4ea9306 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.104555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1327/2623] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:09.105906 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-94cb1c97-c983-4ecc-8429-25b7d4ea9306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:09.108370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8da83c-3a1b-40bc-b320-733dc4df2b63 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.119584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8da83c-3a1b-40bc-b320-733dc4df2b63 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.124176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb1c97-c983-4ecc-8429-25b7d4ea9306 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.125155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe363df-a09f-4f13-b9b6-2633618e3ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.126089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe363df-a09f-4f13-b9b6-2633618e3ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.126434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe363df-a09f-4f13-b9b6-2633618e3ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.126652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe363df-a09f-4f13-b9b6-2633618e3ad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.127414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe363df-a09f-4f13-b9b6-2633618e3ad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:09.127726 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe363df-a09f-4f13-b9b6-2633618e3ad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.130037 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd6795-80c9-41ac-b897-e18dde80eeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.130234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe363df-a09f-4f13-b9b6-2633618e3ad4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.130873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd6795-80c9-41ac-b897-e18dde80eeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.131233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd6795-80c9-41ac-b897-e18dde80eeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.131580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd6795-80c9-41ac-b897-e18dde80eeb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.132482 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd6795-80c9-41ac-b897-e18dde80eeb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.134420 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd6795-80c9-41ac-b897-e18dde80eeb5 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.144916 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1328/2624] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:09.147474 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1297/2625] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:09.151320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe363df-a09f-4f13-b9b6-2633618e3ad4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.151472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd6795-80c9-41ac-b897-e18dde80eeb5 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.162812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe363df-a09f-4f13-b9b6-2633618e3ad4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.167612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd6795-80c9-41ac-b897-e18dde80eeb5 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.169827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dd6a8-28c9-4420-9546-9d978f0ccedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.170670 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dd6a8-28c9-4420-9546-9d978f0ccedf admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.171139 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dd6a8-28c9-4420-9546-9d978f0ccedf admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.171600 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dd6a8-28c9-4420-9546-9d978f0ccedf admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.172502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dd6a8-28c9-4420-9546-9d978f0ccedf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.173042 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b460c42-02aa-4e74-9fa6-ed8419b3b6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.174276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b460c42-02aa-4e74-9fa6-ed8419b3b6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.174792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b460c42-02aa-4e74-9fa6-ed8419b3b6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.175237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b460c42-02aa-4e74-9fa6-ed8419b3b6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.175920 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dd6a8-28c9-4420-9546-9d978f0ccedf admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.176036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b460c42-02aa-4e74-9fa6-ed8419b3b6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.177596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b460c42-02aa-4e74-9fa6-ed8419b3b6a8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.192053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1298/2626] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:09.196202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b460c42-02aa-4e74-9fa6-ed8419b3b6a8 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.209298 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b460c42-02aa-4e74-9fa6-ed8419b3b6a8 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.215123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1329/2627] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:09 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:09.218440 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1215e-3967-46be-9587-fe72aaef6124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.219204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dd6a8-28c9-4420-9546-9d978f0ccedf admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.220784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1215e-3967-46be-9587-fe72aaef6124 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.221110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1215e-3967-46be-9587-fe72aaef6124 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.221821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1215e-3967-46be-9587-fe72aaef6124 admin admin] PATH_INFO: `/v3/projects/11171eb5f1404850bf77aecea367eee1/users/d477bf6de4824d85959d1f9ca05facda/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.244465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1215e-3967-46be-9587-fe72aaef6124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11171eb5f1404850bf77aecea367eee1, user_id=d477bf6de4824d85959d1f9ca05facda, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.247806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1215e-3967-46be-9587-fe72aaef6124 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.277083 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1299/2628] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:09 2022] PUT /identity/v3/projects/11171eb5f1404850bf77aecea367eee1/users/d477bf6de4824d85959d1f9ca05facda/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:09.289930 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9fa37-aa5c-4c73-9f3c-3825c46f6e19 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.290271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9fa37-aa5c-4c73-9f3c-3825c46f6e19 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.290503 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9fa37-aa5c-4c73-9f3c-3825c46f6e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.305598 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dd6a8-28c9-4420-9546-9d978f0ccedf admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.325508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a8f9fa37-aa5c-4c73-9f3c-3825c46f6e19 None None] MFA Rules not processed for user `d477bf6de4824d85959d1f9ca05facda`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:09.327287 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a8f9fa37-aa5c-4c73-9f3c-3825c46f6e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:09.374275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13ac2f-909c-4ea4-9318-89339034b6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.375401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13ac2f-909c-4ea4-9318-89339034b6ef admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.375828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13ac2f-909c-4ea4-9318-89339034b6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.376307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13ac2f-909c-4ea4-9318-89339034b6ef admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.377311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13ac2f-909c-4ea4-9318-89339034b6ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.380240 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13ac2f-909c-4ea4-9318-89339034b6ef admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.411675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1330/2629] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:09 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:09.413076 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13ac2f-909c-4ea4-9318-89339034b6ef admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.414170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1300/2630] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:09.423135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:09.428233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9fa37-aa5c-4c73-9f3c-3825c46f6e19 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.429914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a8f9fa37-aa5c-4c73-9f3c-3825c46f6e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:09.438081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13ac2f-909c-4ea4-9318-89339034b6ef admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.446698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522391a-52af-47ea-91e9-400bfbdf6338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.447693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522391a-52af-47ea-91e9-400bfbdf6338 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.447812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522391a-52af-47ea-91e9-400bfbdf6338 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.448089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522391a-52af-47ea-91e9-400bfbdf6338 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.448867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522391a-52af-47ea-91e9-400bfbdf6338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.452909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522391a-52af-47ea-91e9-400bfbdf6338 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.469203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1331/2631] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:09.470768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522391a-52af-47ea-91e9-400bfbdf6338 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.485077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522391a-52af-47ea-91e9-400bfbdf6338 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.491054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e5e9a-7b48-411f-8ecd-fb5f4ae7c143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.492279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e5e9a-7b48-411f-8ecd-fb5f4ae7c143 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.492863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e5e9a-7b48-411f-8ecd-fb5f4ae7c143 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.493346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e5e9a-7b48-411f-8ecd-fb5f4ae7c143 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.494519 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e5e9a-7b48-411f-8ecd-fb5f4ae7c143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.496622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e5e9a-7b48-411f-8ecd-fb5f4ae7c143 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.513395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1332/2632] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:56:09.514685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e5e9a-7b48-411f-8ecd-fb5f4ae7c143 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.531671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e5e9a-7b48-411f-8ecd-fb5f4ae7c143 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.535553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9fa37-aa5c-4c73-9f3c-3825c46f6e19 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.542340 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dda65-a36b-4a8f-b174-c7b87ed162f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.543603 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0157c11-1bee-4f36-baea-cb30d002a8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.544508 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0157c11-1bee-4f36-baea-cb30d002a8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.544814 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0157c11-1bee-4f36-baea-cb30d002a8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.545331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dda65-a36b-4a8f-b174-c7b87ed162f1 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.545605 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dda65-a36b-4a8f-b174-c7b87ed162f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.545870 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dda65-a36b-4a8f-b174-c7b87ed162f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.546760 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0157c11-1bee-4f36-baea-cb30d002a8c5 admin admin] PATH_INFO: `/v3/projects/6287d9156fb54d7d98cbeb2d3838ade0/users/f454598e31554860a717521fa1158063/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.559250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dda65-a36b-4a8f-b174-c7b87ed162f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.561272 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dda65-a36b-4a8f-b174-c7b87ed162f1 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.571873 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1301/2633] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:09.594874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0157c11-1bee-4f36-baea-cb30d002a8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6287d9156fb54d7d98cbeb2d3838ade0, user_id=f454598e31554860a717521fa1158063, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.594874 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0157c11-1bee-4f36-baea-cb30d002a8c5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.596181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dda65-a36b-4a8f-b174-c7b87ed162f1 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.613784 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:09.620313 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1333/2634] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:09 2022] PUT /identity/v3/projects/6287d9156fb54d7d98cbeb2d3838ade0/users/f454598e31554860a717521fa1158063/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:09.623170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dda65-a36b-4a8f-b174-c7b87ed162f1 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.633341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840932cb-311a-4c9d-9241-b44640bded1b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.633716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840932cb-311a-4c9d-9241-b44640bded1b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.634075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840932cb-311a-4c9d-9241-b44640bded1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.696472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-840932cb-311a-4c9d-9241-b44640bded1b None None] MFA Rules not processed for user `f454598e31554860a717521fa1158063`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:09.700014 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-840932cb-311a-4c9d-9241-b44640bded1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:09.733383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.733383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.733581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.733952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] PATH_INFO: `/v3/users/d477bf6de4824d85959d1f9ca05facda` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.737821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d477bf6de4824d85959d1f9ca05facda)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.742361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.812279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd477bf6de4824d85959d1f9ca05facda'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:09.815798 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1334/2635] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:09.820046 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840932cb-311a-4c9d-9241-b44640bded1b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.831475 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd477bf6de4824d85959d1f9ca05facda'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:09.834743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840932cb-311a-4c9d-9241-b44640bded1b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.842192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5643616-e339-4d00-860a-a2e9e8b77f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.843100 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5643616-e339-4d00-860a-a2e9e8b77f53 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.843551 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5643616-e339-4d00-860a-a2e9e8b77f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.843963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5643616-e339-4d00-860a-a2e9e8b77f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.844876 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5643616-e339-4d00-860a-a2e9e8b77f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.846999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5643616-e339-4d00-860a-a2e9e8b77f53 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.850185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd477bf6de4824d85959d1f9ca05facda'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:09.851236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd477bf6de4824d85959d1f9ca05facda'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:09.859088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-162f23e5-ff0e-44ba-8143-449e9a93d5ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd477bf6de4824d85959d1f9ca05facda'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:09.865734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1302/2636] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:09 2022] DELETE /identity/v3/users/d477bf6de4824d85959d1f9ca05facda => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:09.871029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d3f11-2494-42a7-bcd9-a18442459e97 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.871724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d3f11-2494-42a7-bcd9-a18442459e97 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.872168 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d3f11-2494-42a7-bcd9-a18442459e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.876044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1335/2637] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:09 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:09.879890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5643616-e339-4d00-860a-a2e9e8b77f53 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.881680 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b5643616-e339-4d00-860a-a2e9e8b77f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:09.894201 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c72d3f11-2494-42a7-bcd9-a18442459e97 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:09.895640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c72d3f11-2494-42a7-bcd9-a18442459e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:09.962811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5643616-e339-4d00-860a-a2e9e8b77f53 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:09.970583 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61077abf-6e54-49bd-b6f0-3bbe0034f4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:09.971645 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61077abf-6e54-49bd-b6f0-3bbe0034f4de admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:09.971984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61077abf-6e54-49bd-b6f0-3bbe0034f4de admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:09.972222 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61077abf-6e54-49bd-b6f0-3bbe0034f4de admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:09.972993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61077abf-6e54-49bd-b6f0-3bbe0034f4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:09.974945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61077abf-6e54-49bd-b6f0-3bbe0034f4de admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:09.982023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1303/2638] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:09.992058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d3f11-2494-42a7-bcd9-a18442459e97 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:09.993783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c72d3f11-2494-42a7-bcd9-a18442459e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:10.009074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1336/2639] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:09 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:10.013894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61077abf-6e54-49bd-b6f0-3bbe0034f4de admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:10.030055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61077abf-6e54-49bd-b6f0-3bbe0034f4de admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:10.030853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:10.038681 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060c9e8-67d8-438b-870d-12044e060acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:10.039992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060c9e8-67d8-438b-870d-12044e060acf admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.040788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060c9e8-67d8-438b-870d-12044e060acf admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.041705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060c9e8-67d8-438b-870d-12044e060acf admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.042855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060c9e8-67d8-438b-870d-12044e060acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:10.045755 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060c9e8-67d8-438b-870d-12044e060acf admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:10.070490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1337/2640] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:56:10.077153 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060c9e8-67d8-438b-870d-12044e060acf admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:10.086898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d3f11-2494-42a7-bcd9-a18442459e97 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:10.091797 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:10.092223 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408ffac-fb14-4c3c-99ce-d5e7d3a2b520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:10.093112 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408ffac-fb14-4c3c-99ce-d5e7d3a2b520 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.094100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408ffac-fb14-4c3c-99ce-d5e7d3a2b520 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.094100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408ffac-fb14-4c3c-99ce-d5e7d3a2b520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.102272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060c9e8-67d8-438b-870d-12044e060acf admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:10.114935 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95d9b6-eb6d-4052-b90c-9d73dd13f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:10.116237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408ffac-fb14-4c3c-99ce-d5e7d3a2b520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:10.118310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d9b6-eb6d-4052-b90c-9d73dd13f895 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.119188 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d9b6-eb6d-4052-b90c-9d73dd13f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.120857 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95d9b6-eb6d-4052-b90c-9d73dd13f895 admin admin] PATH_INFO: `/v3/projects/f2fda694cb344e25b1679a897afc5ed7/users/68c580a3d06f4839a04a9bbab6183dcc/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.121152 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408ffac-fb14-4c3c-99ce-d5e7d3a2b520 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:10.135276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1304/2641] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:10.172939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95d9b6-eb6d-4052-b90c-9d73dd13f895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fda694cb344e25b1679a897afc5ed7, user_id=68c580a3d06f4839a04a9bbab6183dcc, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:10.178453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95d9b6-eb6d-4052-b90c-9d73dd13f895 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:10.238889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1338/2642] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:10 2022] PUT /identity/v3/projects/f2fda694cb344e25b1679a897afc5ed7/users/68c580a3d06f4839a04a9bbab6183dcc/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:10.240607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408ffac-fb14-4c3c-99ce-d5e7d3a2b520 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:10.259415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408ffac-fb14-4c3c-99ce-d5e7d3a2b520 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:10.355764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7170281-4221-4d83-940f-9a6e9974f111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:10.357203 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7170281-4221-4d83-940f-9a6e9974f111 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.358019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7170281-4221-4d83-940f-9a6e9974f111 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.358640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7170281-4221-4d83-940f-9a6e9974f111 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.360134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7170281-4221-4d83-940f-9a6e9974f111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:10.363613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7170281-4221-4d83-940f-9a6e9974f111 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:10.386368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1305/2643] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:10.393337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95d9b6-eb6d-4052-b90c-9d73dd13f895 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:10.419061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95d9b6-eb6d-4052-b90c-9d73dd13f895 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:10.429956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545394e-896d-4a38-a431-c81fb17144fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:10.432260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545394e-896d-4a38-a431-c81fb17144fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.432983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545394e-896d-4a38-a431-c81fb17144fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.433589 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545394e-896d-4a38-a431-c81fb17144fc admin admin] PATH_INFO: `/v3/projects/f2fda694cb344e25b1679a897afc5ed7/users/68c580a3d06f4839a04a9bbab6183dcc/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.439266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545394e-896d-4a38-a431-c81fb17144fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fda694cb344e25b1679a897afc5ed7, user_id=68c580a3d06f4839a04a9bbab6183dcc, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:10.445908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545394e-896d-4a38-a431-c81fb17144fc admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:10.473676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1339/2644] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:10 2022] PUT /identity/v3/projects/f2fda694cb344e25b1679a897afc5ed7/users/68c580a3d06f4839a04a9bbab6183dcc/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:10.484904 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae98fa9-4cb1-4262-9eb4-b449c1f26772 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.485460 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae98fa9-4cb1-4262-9eb4-b449c1f26772 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.485817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae98fa9-4cb1-4262-9eb4-b449c1f26772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.503967 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545394e-896d-4a38-a431-c81fb17144fc admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:10.505774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-b545394e-896d-4a38-a431-c81fb17144fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:10.540165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-dae98fa9-4cb1-4262-9eb4-b449c1f26772 None None] MFA Rules not processed for user `68c580a3d06f4839a04a9bbab6183dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:10.542637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-dae98fa9-4cb1-4262-9eb4-b449c1f26772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:10.594485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545394e-896d-4a38-a431-c81fb17144fc admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:10.599873 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:10.601060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.601273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.601544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] PATH_INFO: `/v3/projects/11171eb5f1404850bf77aecea367eee1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.602924 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11171eb5f1404850bf77aecea367eee1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:10.605438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:10.634707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1306/2645] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:10.636092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae98fa9-4cb1-4262-9eb4-b449c1f26772 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:10.691160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11171eb5f1404850bf77aecea367eee1', 'request_id': 'req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:10.706555 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11171eb5f1404850bf77aecea367eee1', 'request_id': 'req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:10.712169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:10.717467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11171eb5f1404850bf77aecea367eee1', 'request_id': 'req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:10.725246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae98fa9-4cb1-4262-9eb4-b449c1f26772 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:10.731069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] The token cache is being invalidate because project 11171eb5f1404850bf77aecea367eee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:10.731362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:10.732151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:10.734325 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1340/2646] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:10 2022] DELETE /identity/v3/projects/11171eb5f1404850bf77aecea367eee1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:10.734891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:10.736359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:10.825009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0724e24-6cbd-4a56-b468-3524bf58d6e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:10.825606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0724e24-6cbd-4a56-b468-3524bf58d6e7 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.829457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64dbe1a-ed13-4dc2-8e34-4734a3d99817 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:10.832072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0724e24-6cbd-4a56-b468-3524bf58d6e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.832430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0724e24-6cbd-4a56-b468-3524bf58d6e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.833618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:10.834490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.834716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f0724e24-6cbd-4a56-b468-3524bf58d6e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:10.834899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.835247 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.839036 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:10.856257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:10.944227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0724e24-6cbd-4a56-b468-3524bf58d6e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:10.945625 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0724e24-6cbd-4a56-b468-3524bf58d6e7 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:10.954649 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:10.960628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:10.980842 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1307/2647] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:10.982291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19df6e-501d-4abd-a74a-0cfaf169f3a2 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:10.982578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19df6e-501d-4abd-a74a-0cfaf169f3a2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:10.982863 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19df6e-501d-4abd-a74a-0cfaf169f3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:10.987119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1341/2648] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:11.061368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8f19df6e-501d-4abd-a74a-0cfaf169f3a2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:11.061368 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8f19df6e-501d-4abd-a74a-0cfaf169f3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.143917 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.151339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.157642 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43656b9-f0d7-4fb1-8116-cb2db3ebdafc service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:11.166435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:11.166435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.166435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.166435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.167618 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.209565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1308/2649] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:11.219780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431533a1-4d9c-466d-9582-1c315aef6e0c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.220131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431533a1-4d9c-466d-9582-1c315aef6e0c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.221505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431533a1-4d9c-466d-9582-1c315aef6e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.247621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-431533a1-4d9c-466d-9582-1c315aef6e0c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:11.248970 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-431533a1-4d9c-466d-9582-1c315aef6e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.258547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1309/2650] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:11.263879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431533a1-4d9c-466d-9582-1c315aef6e0c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.268137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:11.270042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:11.275835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431533a1-4d9c-466d-9582-1c315aef6e0c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:11.281215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225a517-46ac-4d26-a1b2-9cc7ca7cfcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:11.281993 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225a517-46ac-4d26-a1b2-9cc7ca7cfcdd admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.282391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225a517-46ac-4d26-a1b2-9cc7ca7cfcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.282956 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225a517-46ac-4d26-a1b2-9cc7ca7cfcdd admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.284040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1342/2651] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:11.285460 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.287096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.295648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225a517-46ac-4d26-a1b2-9cc7ca7cfcdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:11.296071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225a517-46ac-4d26-a1b2-9cc7ca7cfcdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:11.305256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225a517-46ac-4d26-a1b2-9cc7ca7cfcdd admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:11.309909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3df28c-3bbf-4f7e-a94b-5a72038aafde service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:11.315079 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:11.316235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.316235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.316919 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] PATH_INFO: `/v3/users/2e78cf97bac4484f93c7359b86a26628` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.317838 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e78cf97bac4484f93c7359b86a26628)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:11.319775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:11.338325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1310/2652] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:56:11.339702 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225a517-46ac-4d26-a1b2-9cc7ca7cfcdd admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.371045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225a517-46ac-4d26-a1b2-9cc7ca7cfcdd admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:11.386875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418b716-4eca-44bb-8215-56cc2b4d2b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:11.388103 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418b716-4eca-44bb-8215-56cc2b4d2b70 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.388504 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418b716-4eca-44bb-8215-56cc2b4d2b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.389108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418b716-4eca-44bb-8215-56cc2b4d2b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.390646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418b716-4eca-44bb-8215-56cc2b4d2b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:11.392670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418b716-4eca-44bb-8215-56cc2b4d2b70 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:11.398829 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e78cf97bac4484f93c7359b86a26628'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.418351 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e78cf97bac4484f93c7359b86a26628'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.420382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e78cf97bac4484f93c7359b86a26628'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.421201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e78cf97bac4484f93c7359b86a26628'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.435276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e78cf97bac4484f93c7359b86a26628'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.438545 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1311/2653] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:11 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:11.443815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418b716-4eca-44bb-8215-56cc2b4d2b70 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.446154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-2418b716-4eca-44bb-8215-56cc2b4d2b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.459274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1343/2654] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:11 2022] DELETE /identity/v3/users/2e78cf97bac4484f93c7359b86a26628 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:11.477321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:11.555660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418b716-4eca-44bb-8215-56cc2b4d2b70 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:11.643755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dc94d-d9c3-4941-ac25-ab523559997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:11.644849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dc94d-d9c3-4941-ac25-ab523559997d admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.648928 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dc94d-d9c3-4941-ac25-ab523559997d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.649386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886dc94d-d9c3-4941-ac25-ab523559997d admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.652181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dc94d-d9c3-4941-ac25-ab523559997d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:11.653914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886dc94d-d9c3-4941-ac25-ab523559997d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:11.683945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.685887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.695132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1312/2655] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:11 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:11.696224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dc94d-d9c3-4941-ac25-ab523559997d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.708554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4961952-4294-4fb7-9104-4484d5165464 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:11.710823 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886dc94d-d9c3-4941-ac25-ab523559997d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:11.714995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:11.716105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.716421 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.716694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] PATH_INFO: `/v3/users/f454598e31554860a717521fa1158063` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.718303 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f454598e31554860a717521fa1158063)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:11.719048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72a083-ec5e-4377-a450-ac689d87b934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:11.719795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72a083-ec5e-4377-a450-ac689d87b934 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.720169 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72a083-ec5e-4377-a450-ac689d87b934 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.720507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72a083-ec5e-4377-a450-ac689d87b934 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.720751 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:11.721709 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72a083-ec5e-4377-a450-ac689d87b934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:11.723661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72a083-ec5e-4377-a450-ac689d87b934 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:11.738929 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1313/2656] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:11.740094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72a083-ec5e-4377-a450-ac689d87b934 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.747656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1d72a083-ec5e-4377-a450-ac689d87b934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.751420 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:11.764947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72a083-ec5e-4377-a450-ac689d87b934 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:11.770731 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:11.771648 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.772267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.772787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] PATH_INFO: `/v3/projects/5ccf88e235fa45d1b92c7c5612bfe140` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.774177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ccf88e235fa45d1b92c7c5612bfe140)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:11.777597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:11.797082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f454598e31554860a717521fa1158063'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.815658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f454598e31554860a717521fa1158063'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.816802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f454598e31554860a717521fa1158063'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.817515 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f454598e31554860a717521fa1158063'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.826309 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f454598e31554860a717521fa1158063'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.833333 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ccf88e235fa45d1b92c7c5612bfe140', 'request_id': 'req-33c13b91-78f0-42db-81b1-f771bef6ec92'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.840060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1344/2657] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:11 2022] DELETE /identity/v3/users/f454598e31554860a717521fa1158063 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:11.843145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.843145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.843334 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ccf88e235fa45d1b92c7c5612bfe140', 'request_id': 'req-33c13b91-78f0-42db-81b1-f771bef6ec92'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.851407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ccf88e235fa45d1b92c7c5612bfe140', 'request_id': 'req-33c13b91-78f0-42db-81b1-f771bef6ec92'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.865798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] The token cache is being invalidate because project 5ccf88e235fa45d1b92c7c5612bfe140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:11.866084 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.866755 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-33c13b91-78f0-42db-81b1-f771bef6ec92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:11.868636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1314/2658] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:11 2022] DELETE /identity/v3/projects/5ccf88e235fa45d1b92c7c5612bfe140 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:11.874069 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf3eef-0b7c-42e8-810b-dea638cc38a2 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.874358 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf3eef-0b7c-42e8-810b-dea638cc38a2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.875022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf3eef-0b7c-42e8-810b-dea638cc38a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.895024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-37cf3eef-0b7c-42e8-810b-dea638cc38a2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:11.896848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-37cf3eef-0b7c-42e8-810b-dea638cc38a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.919004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:11.921772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78a765-e6f4-45ce-9415-7a0a6d3bfa54 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:11.926846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f595ba-e811-4de8-ba74-7b2592dab1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:11.927590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f595ba-e811-4de8-ba74-7b2592dab1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.927939 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f595ba-e811-4de8-ba74-7b2592dab1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.928287 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f595ba-e811-4de8-ba74-7b2592dab1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.929128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f595ba-e811-4de8-ba74-7b2592dab1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:11.930733 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f595ba-e811-4de8-ba74-7b2592dab1b3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:11.947404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1345/2659] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:11.948750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b804a-f349-41ef-b67f-55447a07fd09 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:11.949042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b804a-f349-41ef-b67f-55447a07fd09 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:11.949320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b804a-f349-41ef-b67f-55447a07fd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:11.968945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0e8b804a-f349-41ef-b67f-55447a07fd09 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:11.970311 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0e8b804a-f349-41ef-b67f-55447a07fd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.976634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1315/2660] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:11.976634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf3eef-0b7c-42e8-810b-dea638cc38a2 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.981322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-37cf3eef-0b7c-42e8-810b-dea638cc38a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.981949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1346/2661] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:11.992700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b804a-f349-41ef-b67f-55447a07fd09 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:11.995110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0e8b804a-f349-41ef-b67f-55447a07fd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:11.999570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf3eef-0b7c-42e8-810b-dea638cc38a2 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:12.008250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487398c-f5e6-4ed4-bb13-0af68b4244f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:12.009288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487398c-f5e6-4ed4-bb13-0af68b4244f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.009636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487398c-f5e6-4ed4-bb13-0af68b4244f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.010892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487398c-f5e6-4ed4-bb13-0af68b4244f0 admin admin] PATH_INFO: `/v3/projects/9c6c8c69f9cb4c4ca5b4e71340689bad/users/b57e31d4bdaf41059c622fb91d3a703e/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.029676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487398c-f5e6-4ed4-bb13-0af68b4244f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6c8c69f9cb4c4ca5b4e71340689bad, user_id=b57e31d4bdaf41059c622fb91d3a703e, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:12.034235 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487398c-f5e6-4ed4-bb13-0af68b4244f0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:12.053155 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1316/2662] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:11 2022] PUT /identity/v3/projects/9c6c8c69f9cb4c4ca5b4e71340689bad/users/b57e31d4bdaf41059c622fb91d3a703e/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:12.054442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423596a3-b1ce-4ac5-addf-84ffaf109505 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.054742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423596a3-b1ce-4ac5-addf-84ffaf109505 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.055002 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423596a3-b1ce-4ac5-addf-84ffaf109505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.067571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b804a-f349-41ef-b67f-55447a07fd09 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:12.075674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-423596a3-b1ce-4ac5-addf-84ffaf109505 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:12.076935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-423596a3-b1ce-4ac5-addf-84ffaf109505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:12.109885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:12.161767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa858b1-4342-4328-bcf7-c1f415bd2888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:12.162707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa858b1-4342-4328-bcf7-c1f415bd2888 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.163144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa858b1-4342-4328-bcf7-c1f415bd2888 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.163597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa858b1-4342-4328-bcf7-c1f415bd2888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.210456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1317/2663] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:12.212153 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca9112-a644-4e42-9192-c2d1fca973a5 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.212646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca9112-a644-4e42-9192-c2d1fca973a5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.213331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca9112-a644-4e42-9192-c2d1fca973a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.265767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa858b1-4342-4328-bcf7-c1f415bd2888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:12.267257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa858b1-4342-4328-bcf7-c1f415bd2888 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:12.279344 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1347/2664] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:12.280237 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa858b1-4342-4328-bcf7-c1f415bd2888 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:12.281245 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-faa858b1-4342-4328-bcf7-c1f415bd2888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:12.284057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-e0ca9112-a644-4e42-9192-c2d1fca973a5 None None] MFA Rules not processed for user `b57e31d4bdaf41059c622fb91d3a703e`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:12.285445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e0ca9112-a644-4e42-9192-c2d1fca973a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:12.297651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa858b1-4342-4328-bcf7-c1f415bd2888 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:12.306467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:12.308038 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.308319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.308613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] PATH_INFO: `/v3/users/fdd3aa5da8a74c0a85d3363dc211257d` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.309856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd3aa5da8a74c0a85d3363dc211257d)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:12.312419 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:12.399513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd3aa5da8a74c0a85d3363dc211257d'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:12.425339 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd3aa5da8a74c0a85d3363dc211257d'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:12.426703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd3aa5da8a74c0a85d3363dc211257d'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:12.428989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd3aa5da8a74c0a85d3363dc211257d'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:12.446362 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd3aa5da8a74c0a85d3363dc211257d'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:12.458935 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1318/2665] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:12.459817 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca9112-a644-4e42-9192-c2d1fca973a5 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:12.471828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1348/2666] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:12 2022] DELETE /identity/v3/users/fdd3aa5da8a74c0a85d3363dc211257d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:12.472488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:12.475597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:12.483431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca9112-a644-4e42-9192-c2d1fca973a5 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:12.530763 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:12.567150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88452c-fc61-4683-888f-64645f6922fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:12.568170 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88452c-fc61-4683-888f-64645f6922fe admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.568463 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88452c-fc61-4683-888f-64645f6922fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.568793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88452c-fc61-4683-888f-64645f6922fe admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.569649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88452c-fc61-4683-888f-64645f6922fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:12.569968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88452c-fc61-4683-888f-64645f6922fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:12.571802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88452c-fc61-4683-888f-64645f6922fe admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:12.581088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f865d9b-4826-4cc4-9342-701f932adebe admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:12.588402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1319/2667] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:12.589805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251b862-3e6f-4324-9fdc-e3fd100aace5 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.590105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251b862-3e6f-4324-9fdc-e3fd100aace5 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.590390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251b862-3e6f-4324-9fdc-e3fd100aace5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.594006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cb95f-c6cd-4975-8ab9-4938df7072c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:12.594992 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb95f-c6cd-4975-8ab9-4938df7072c1 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.595439 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb95f-c6cd-4975-8ab9-4938df7072c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.595865 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73cb95f-c6cd-4975-8ab9-4938df7072c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.600081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c73cb95f-c6cd-4975-8ab9-4938df7072c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:12.613419 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-c251b862-3e6f-4324-9fdc-e3fd100aace5 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:12.615585 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c251b862-3e6f-4324-9fdc-e3fd100aace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:12.626173 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1320/2668] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:12.627505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:12.629000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251b862-3e6f-4324-9fdc-e3fd100aace5 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:12.631140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c251b862-3e6f-4324-9fdc-e3fd100aace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:12.654552 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251b862-3e6f-4324-9fdc-e3fd100aace5 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:12.661049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248201a1-c41c-4da0-afed-113712357f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:12.662341 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248201a1-c41c-4da0-afed-113712357f87 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.662720 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248201a1-c41c-4da0-afed-113712357f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.663093 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248201a1-c41c-4da0-afed-113712357f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.664546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248201a1-c41c-4da0-afed-113712357f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:12.667148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248201a1-c41c-4da0-afed-113712357f87 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:12.707345 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1321/2669] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:12 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:12.708512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248201a1-c41c-4da0-afed-113712357f87 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:12.711384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-248201a1-c41c-4da0-afed-113712357f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:12.741623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cb95f-c6cd-4975-8ab9-4938df7072c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:12.743393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73cb95f-c6cd-4975-8ab9-4938df7072c1 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:12.757637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1349/2670] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:12.762230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cb95f-c6cd-4975-8ab9-4938df7072c1 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:12.783001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73cb95f-c6cd-4975-8ab9-4938df7072c1 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:12.805678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248201a1-c41c-4da0-afed-113712357f87 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:12.817923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5afbc4-fe72-4902-81a3-39428a5d395f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:12.818756 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5afbc4-fe72-4902-81a3-39428a5d395f service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.819186 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5afbc4-fe72-4902-81a3-39428a5d395f service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.819563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5afbc4-fe72-4902-81a3-39428a5d395f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.900110 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515da8e-c2b9-4e59-897d-82b400749a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:12.901367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515da8e-c2b9-4e59-897d-82b400749a03 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:12.902090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515da8e-c2b9-4e59-897d-82b400749a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:12.903893 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515da8e-c2b9-4e59-897d-82b400749a03 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:12.906236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515da8e-c2b9-4e59-897d-82b400749a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:12.908335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515da8e-c2b9-4e59-897d-82b400749a03 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:12.965558 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5afbc4-fe72-4902-81a3-39428a5d395f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:12.970694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5afbc4-fe72-4902-81a3-39428a5d395f service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:12.975087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1350/2671] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:12 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:12.980254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515da8e-c2b9-4e59-897d-82b400749a03 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:12.992904 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515da8e-c2b9-4e59-897d-82b400749a03 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:13.002651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1322/2672] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:13.004794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb62c93-01ad-4c56-ab91-e8103c5177ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:13.005648 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb62c93-01ad-4c56-ab91-e8103c5177ae admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:13.005926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb62c93-01ad-4c56-ab91-e8103c5177ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:13.006242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb62c93-01ad-4c56-ab91-e8103c5177ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:13.006991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb62c93-01ad-4c56-ab91-e8103c5177ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:13.008699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb62c93-01ad-4c56-ab91-e8103c5177ae admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:13.041963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1351/2673] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:13.042851 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5afbc4-fe72-4902-81a3-39428a5d395f service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:13.053539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5afbc4-fe72-4902-81a3-39428a5d395f service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:13.060835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d225a-6134-4f8f-95c8-40d5e6c03b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:13.061596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d225a-6134-4f8f-95c8-40d5e6c03b91 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:13.061875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d225a-6134-4f8f-95c8-40d5e6c03b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:13.062159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d225a-6134-4f8f-95c8-40d5e6c03b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:13.062931 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d225a-6134-4f8f-95c8-40d5e6c03b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:13.064859 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d225a-6134-4f8f-95c8-40d5e6c03b91 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:13.085044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1323/2674] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:56:13.086271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb62c93-01ad-4c56-ab91-e8103c5177ae admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:13.102663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb62c93-01ad-4c56-ab91-e8103c5177ae admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:13.132518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b57a2-6853-4e01-8779-b8718400e5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:13.133502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b57a2-6853-4e01-8779-b8718400e5c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:13.133813 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b57a2-6853-4e01-8779-b8718400e5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:13.134092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b57a2-6853-4e01-8779-b8718400e5c1 admin admin] PATH_INFO: `/v3/projects/46f447736b9243bf96804fdbe7e194da/users/8204dee5ba6e4049b0bb25f0389cb4e8/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:13.168831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d225a-6134-4f8f-95c8-40d5e6c03b91 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:13.170854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8e4d225a-6134-4f8f-95c8-40d5e6c03b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:13.178710 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b57a2-6853-4e01-8779-b8718400e5c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f447736b9243bf96804fdbe7e194da, user_id=8204dee5ba6e4049b0bb25f0389cb4e8, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:13.186757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b57a2-6853-4e01-8779-b8718400e5c1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:13.206516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d225a-6134-4f8f-95c8-40d5e6c03b91 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:13.216721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1352/2675] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:13 2022] PUT /identity/v3/projects/46f447736b9243bf96804fdbe7e194da/users/8204dee5ba6e4049b0bb25f0389cb4e8/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:13.235104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6117b-f1af-40e1-b36a-bdc2fd111a3d None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:13.235309 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6117b-f1af-40e1-b36a-bdc2fd111a3d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:13.235520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6117b-f1af-40e1-b36a-bdc2fd111a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:13.281030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:13.289415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1fd6117b-f1af-40e1-b36a-bdc2fd111a3d None None] MFA Rules not processed for user `8204dee5ba6e4049b0bb25f0389cb4e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:13.290825 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1fd6117b-f1af-40e1-b36a-bdc2fd111a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:13.321120 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:13.322275 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:13.322559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:13.323958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] PATH_INFO: `/v3/projects/6287d9156fb54d7d98cbeb2d3838ade0` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:13.325586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6287d9156fb54d7d98cbeb2d3838ade0)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:13.328115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:13.382401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:13.397799 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1353/2676] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:13.401776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6287d9156fb54d7d98cbeb2d3838ade0', 'request_id': 'req-58934157-0685-4bd8-ad84-71b3535fe39f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.409467 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6117b-f1af-40e1-b36a-bdc2fd111a3d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:13.411195 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6287d9156fb54d7d98cbeb2d3838ade0', 'request_id': 'req-58934157-0685-4bd8-ad84-71b3535fe39f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.411355 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1fd6117b-f1af-40e1-b36a-bdc2fd111a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:13.420304 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6287d9156fb54d7d98cbeb2d3838ade0', 'request_id': 'req-58934157-0685-4bd8-ad84-71b3535fe39f'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.433313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] The token cache is being invalidate because project 6287d9156fb54d7d98cbeb2d3838ade0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:13.433621 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.434386 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.436253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1324/2677] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:13 2022] DELETE /identity/v3/projects/6287d9156fb54d7d98cbeb2d3838ade0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:13.439927 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:13.442315 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:13.506359 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6117b-f1af-40e1-b36a-bdc2fd111a3d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:13.568057 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58934157-0685-4bd8-ad84-71b3535fe39f admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:13.574513 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:13.575564 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:13.575945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:13.576318 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] PATH_INFO: `/v3/users/68c580a3d06f4839a04a9bbab6183dcc` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:13.577751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c580a3d06f4839a04a9bbab6183dcc)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:13.579764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:13.589950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:13.590558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:13.590844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:13.591242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:13.593048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:13.646044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c580a3d06f4839a04a9bbab6183dcc'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.662248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:13.663880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:13.667707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c580a3d06f4839a04a9bbab6183dcc'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.671660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:13.676042 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1354/2678] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:13.677002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:13.678411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:13.679102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c580a3d06f4839a04a9bbab6183dcc'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.680688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c580a3d06f4839a04a9bbab6183dcc'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.695125 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c580a3d06f4839a04a9bbab6183dcc'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.698370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268147-6f91-499f-a41b-de181fc16216 service nova] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:13.703946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:13.705003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:13.705736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:13.705944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1325/2679] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:13 2022] DELETE /identity/v3/users/68c580a3d06f4839a04a9bbab6183dcc => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:13.710220 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] PATH_INFO: `/v3/projects/40fd86bfacae4cb6adb22f54962238d6` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:13.720293 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fd86bfacae4cb6adb22f54962238d6)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:13.722657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:13.768440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:13.794280 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fd86bfacae4cb6adb22f54962238d6', 'request_id': 'req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.813881 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fd86bfacae4cb6adb22f54962238d6', 'request_id': 'req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.823997 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fd86bfacae4cb6adb22f54962238d6', 'request_id': 'req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.837242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] The token cache is being invalidate because project 40fd86bfacae4cb6adb22f54962238d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:13.837654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.839154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e0f45c61-43cd-43c7-aeaa-3872ec351a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:13.842923 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1355/2680] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:13 2022] DELETE /identity/v3/projects/40fd86bfacae4cb6adb22f54962238d6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:14.140612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:14.141974 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:14.273005 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e17ecf-f1e0-4aa7-ae47-390965b328c7 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:14.347199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:14.349177 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:14.349654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:14.350145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] PATH_INFO: `/v3/projects/f2fda694cb344e25b1679a897afc5ed7` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:14.352354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fda694cb344e25b1679a897afc5ed7)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:14.355743 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:14.416890 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fda694cb344e25b1679a897afc5ed7', 'request_id': 'req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:14.424696 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fda694cb344e25b1679a897afc5ed7', 'request_id': 'req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:14.435100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fda694cb344e25b1679a897afc5ed7', 'request_id': 'req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:14.457039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] The token cache is being invalidate because project f2fda694cb344e25b1679a897afc5ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:14.457452 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:14.458676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-eb1c21c0-0bc0-4c55-ac00-4949a176ec95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:14.460215 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1326/2681] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:14 2022] DELETE /identity/v3/projects/f2fda694cb344e25b1679a897afc5ed7 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:14.468998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4021c8-945f-42d9-8ab6-2f18e8039716 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:14.469457 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4021c8-945f-42d9-8ab6-2f18e8039716 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:14.469856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4021c8-945f-42d9-8ab6-2f18e8039716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:14.491037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-cb4021c8-945f-42d9-8ab6-2f18e8039716 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:14.492299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-cb4021c8-945f-42d9-8ab6-2f18e8039716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:14.566697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1356/2682] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:14.574744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67ac4f-50ba-4ae1-9a0b-4f1cfbd580c1 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:14.575226 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67ac4f-50ba-4ae1-9a0b-4f1cfbd580c1 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:14.575316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67ac4f-50ba-4ae1-9a0b-4f1cfbd580c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:14.595374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8b67ac4f-50ba-4ae1-9a0b-4f1cfbd580c1 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:14.600490 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8b67ac4f-50ba-4ae1-9a0b-4f1cfbd580c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:14.605265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1327/2683] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:14.609196 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4021c8-945f-42d9-8ab6-2f18e8039716 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:14.619251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4021c8-945f-42d9-8ab6-2f18e8039716 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:14.623966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36ed58-75c4-4ce4-8fca-d4addd730ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:14.624663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36ed58-75c4-4ce4-8fca-d4addd730ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:14.624949 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36ed58-75c4-4ce4-8fca-d4addd730ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:14.625226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36ed58-75c4-4ce4-8fca-d4addd730ff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:14.625957 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36ed58-75c4-4ce4-8fca-d4addd730ff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:14.626266 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36ed58-75c4-4ce4-8fca-d4addd730ff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:14.627847 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36ed58-75c4-4ce4-8fca-d4addd730ff9 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:14.645488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1357/2684] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:14.646311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67ac4f-50ba-4ae1-9a0b-4f1cfbd580c1 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:14.658697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67ac4f-50ba-4ae1-9a0b-4f1cfbd580c1 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:14.668989 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848379b-45a9-4d04-b36b-ecec72d8c598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:14.669848 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848379b-45a9-4d04-b36b-ecec72d8c598 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:14.670224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848379b-45a9-4d04-b36b-ecec72d8c598 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:14.670585 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848379b-45a9-4d04-b36b-ecec72d8c598 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:14.671406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848379b-45a9-4d04-b36b-ecec72d8c598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:14.673437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848379b-45a9-4d04-b36b-ecec72d8c598 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:14.688543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:14.715259 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1328/2685] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:14 2022] POST /identity/v3/projects => generated 361 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:14.717545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36ed58-75c4-4ce4-8fca-d4addd730ff9 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:14.727742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36ed58-75c4-4ce4-8fca-d4addd730ff9 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:14.823033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ad45e-8bc7-4cf9-8433-0f453e0317be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:14.823713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ad45e-8bc7-4cf9-8433-0f453e0317be admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:14.824048 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ad45e-8bc7-4cf9-8433-0f453e0317be admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:14.824267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ad45e-8bc7-4cf9-8433-0f453e0317be admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:14.825080 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ad45e-8bc7-4cf9-8433-0f453e0317be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:14.827155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ad45e-8bc7-4cf9-8433-0f453e0317be admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:14.865320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1358/2686] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:14 2022] POST /identity/v3/users => generated 281 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:14.870489 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848379b-45a9-4d04-b36b-ecec72d8c598 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:14.881746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848379b-45a9-4d04-b36b-ecec72d8c598 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:14.888558 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e493d1-d3e2-4b5c-8614-17afac736dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:14.889347 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e493d1-d3e2-4b5c-8614-17afac736dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:14.889634 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e493d1-d3e2-4b5c-8614-17afac736dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:14.889968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e493d1-d3e2-4b5c-8614-17afac736dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:14.890703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e493d1-d3e2-4b5c-8614-17afac736dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:14.892395 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e493d1-d3e2-4b5c-8614-17afac736dc5 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:14.905294 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1329/2687] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:14.909507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ad45e-8bc7-4cf9-8433-0f453e0317be admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:14.920394 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ad45e-8bc7-4cf9-8433-0f453e0317be admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:14.925411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413736ac-f2eb-4409-be00-301ba88145ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:14.926082 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413736ac-f2eb-4409-be00-301ba88145ef admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:14.926370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413736ac-f2eb-4409-be00-301ba88145ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:14.926650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413736ac-f2eb-4409-be00-301ba88145ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:14.927383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413736ac-f2eb-4409-be00-301ba88145ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:14.929234 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413736ac-f2eb-4409-be00-301ba88145ef admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:14.942229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1359/2688] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:14.946459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e493d1-d3e2-4b5c-8614-17afac736dc5 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:14.957623 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e493d1-d3e2-4b5c-8614-17afac736dc5 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:14.962449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f026089-0215-4a47-8bf4-4b3f9c94b336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:14.963430 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f026089-0215-4a47-8bf4-4b3f9c94b336 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:14.963715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f026089-0215-4a47-8bf4-4b3f9c94b336 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:14.964007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f026089-0215-4a47-8bf4-4b3f9c94b336 admin admin] PATH_INFO: `/v3/projects/169398b6cf6d4b0fa927fd39f044056c/users/c2965d9def8841e78376e40c30e587b7/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:14.980525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f026089-0215-4a47-8bf4-4b3f9c94b336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169398b6cf6d4b0fa927fd39f044056c, user_id=c2965d9def8841e78376e40c30e587b7, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:14.983250 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f026089-0215-4a47-8bf4-4b3f9c94b336 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:14.998141 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1330/2689] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:14 2022] PUT /identity/v3/projects/169398b6cf6d4b0fa927fd39f044056c/users/c2965d9def8841e78376e40c30e587b7/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:15.009684 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15243be9-ecca-4b0e-a13f-0dc0d7df9b82 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:15.012096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15243be9-ecca-4b0e-a13f-0dc0d7df9b82 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:15.012096 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15243be9-ecca-4b0e-a13f-0dc0d7df9b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:15.042861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-15243be9-ecca-4b0e-a13f-0dc0d7df9b82 None None] MFA Rules not processed for user `c2965d9def8841e78376e40c30e587b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:15.044081 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-15243be9-ecca-4b0e-a13f-0dc0d7df9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:15.054438 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:15.141937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1360/2690] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:15.171389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f026089-0215-4a47-8bf4-4b3f9c94b336 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:15.173400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1f026089-0215-4a47-8bf4-4b3f9c94b336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:15.258853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f026089-0215-4a47-8bf4-4b3f9c94b336 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:15.263437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa626af0-9405-48d1-9a35-d22226024ed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:15.264110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa626af0-9405-48d1-9a35-d22226024ed2 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:15.264400 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa626af0-9405-48d1-9a35-d22226024ed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:15.264775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa626af0-9405-48d1-9a35-d22226024ed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:15.276828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa626af0-9405-48d1-9a35-d22226024ed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:15.278291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa626af0-9405-48d1-9a35-d22226024ed2 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:15.289325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1331/2691] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:15.305815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15243be9-ecca-4b0e-a13f-0dc0d7df9b82 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:15.390601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15243be9-ecca-4b0e-a13f-0dc0d7df9b82 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:15.396441 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:15.397399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:15.397719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:15.398135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] PATH_INFO: `/v3/users/c2965d9def8841e78376e40c30e587b7` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:15.399343 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2965d9def8841e78376e40c30e587b7)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:15.403235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:15.460565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2965d9def8841e78376e40c30e587b7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:15.472891 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2965d9def8841e78376e40c30e587b7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:15.473617 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2965d9def8841e78376e40c30e587b7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:15.474248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2965d9def8841e78376e40c30e587b7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:15.481749 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2965d9def8841e78376e40c30e587b7'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:15.497622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1361/2692] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:15 2022] DELETE /identity/v3/users/c2965d9def8841e78376e40c30e587b7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:15.502462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abe500-57c4-4d27-ba17-51bb41e5e418 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:15.502821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abe500-57c4-4d27-ba17-51bb41e5e418 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:15.503202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abe500-57c4-4d27-ba17-51bb41e5e418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:15.523378 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-d3abe500-57c4-4d27-ba17-51bb41e5e418 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:15.524753 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d3abe500-57c4-4d27-ba17-51bb41e5e418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:15.604187 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1332/2693] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:15.618569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:15.620424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:15.659002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:15.692345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695f59e-9be5-4180-ad81-af3f7ff59b29 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:15.698357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638daf2d-9ca3-4fd4-b13a-fe4ceef61030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:15.699298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638daf2d-9ca3-4fd4-b13a-fe4ceef61030 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:15.699715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638daf2d-9ca3-4fd4-b13a-fe4ceef61030 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:15.700122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638daf2d-9ca3-4fd4-b13a-fe4ceef61030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:15.718300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638daf2d-9ca3-4fd4-b13a-fe4ceef61030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:15.719782 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638daf2d-9ca3-4fd4-b13a-fe4ceef61030 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:15.734244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1362/2694] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:16.005212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abe500-57c4-4d27-ba17-51bb41e5e418 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.006651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-d3abe500-57c4-4d27-ba17-51bb41e5e418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.020407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abe500-57c4-4d27-ba17-51bb41e5e418 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.025862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.026844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.027138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.027421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] PATH_INFO: `/v3/projects/169398b6cf6d4b0fa927fd39f044056c` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.028684 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] RBAC: Authorizing `identity:delete_project(project_id=169398b6cf6d4b0fa927fd39f044056c)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.030609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.058963 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638daf2d-9ca3-4fd4-b13a-fe4ceef61030 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.060791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-638daf2d-9ca3-4fd4-b13a-fe4ceef61030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.074415 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638daf2d-9ca3-4fd4-b13a-fe4ceef61030 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.080241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169398b6cf6d4b0fa927fd39f044056c', 'request_id': 'req-39f478cb-8fd5-4d82-b181-bfad938aafee'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.087598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169398b6cf6d4b0fa927fd39f044056c', 'request_id': 'req-39f478cb-8fd5-4d82-b181-bfad938aafee'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.092281 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:16.097910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169398b6cf6d4b0fa927fd39f044056c', 'request_id': 'req-39f478cb-8fd5-4d82-b181-bfad938aafee'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.111981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] The token cache is being invalidate because project 169398b6cf6d4b0fa927fd39f044056c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:16.112273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.112936 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-39f478cb-8fd5-4d82-b181-bfad938aafee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.114517 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1333/2695] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:16 2022] DELETE /identity/v3/projects/169398b6cf6d4b0fa927fd39f044056c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:16.119342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e432b39-c433-4077-9827-09e5ac24167e None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.119633 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e432b39-c433-4077-9827-09e5ac24167e None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.119910 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e432b39-c433-4077-9827-09e5ac24167e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.133105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.134059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.134346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.135141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] PATH_INFO: `/v3/users/b57e31d4bdaf41059c622fb91d3a703e` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.136490 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b57e31d4bdaf41059c622fb91d3a703e)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.138424 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.138744 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-6e432b39-c433-4077-9827-09e5ac24167e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:16.139962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6e432b39-c433-4077-9827-09e5ac24167e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.148331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1334/2696] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:16.158485 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e94cd-46c0-4dcb-845b-662455fa5f1d None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.158776 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e94cd-46c0-4dcb-845b-662455fa5f1d None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.159059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e94cd-46c0-4dcb-845b-662455fa5f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.178457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-7c5e94cd-46c0-4dcb-845b-662455fa5f1d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:16.179747 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7c5e94cd-46c0-4dcb-845b-662455fa5f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.190154 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1335/2697] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:16.193580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e94cd-46c0-4dcb-845b-662455fa5f1d None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.200494 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b57e31d4bdaf41059c622fb91d3a703e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.203971 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e94cd-46c0-4dcb-845b-662455fa5f1d None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.208955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.209618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.209902 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.210172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.210948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:16.211260 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.212961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.214327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b57e31d4bdaf41059c622fb91d3a703e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.215112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b57e31d4bdaf41059c622fb91d3a703e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.216019 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b57e31d4bdaf41059c622fb91d3a703e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.225192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-65e066ba-93ff-4bc7-bb91-cea0b6f0fce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b57e31d4bdaf41059c622fb91d3a703e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:16.225683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1336/2698] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:56:16.230044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.231771 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.239446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1363/2699] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:16 2022] DELETE /identity/v3/users/b57e31d4bdaf41059c622fb91d3a703e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:16.243273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfd524-0d36-49cb-92bf-293ec4785eda None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.243560 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfd524-0d36-49cb-92bf-293ec4785eda None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.243837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfd524-0d36-49cb-92bf-293ec4785eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.247017 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559796-cba6-4dae-9f2e-c4cea2a6592a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.263440 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-21dfd524-0d36-49cb-92bf-293ec4785eda None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:16.264818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-21dfd524-0d36-49cb-92bf-293ec4785eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.314976 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:16.318637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5025b-2af8-41be-86f2-796b2b837d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.319316 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5025b-2af8-41be-86f2-796b2b837d03 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.319607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5025b-2af8-41be-86f2-796b2b837d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.319933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5025b-2af8-41be-86f2-796b2b837d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.320612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5025b-2af8-41be-86f2-796b2b837d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.323136 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5025b-2af8-41be-86f2-796b2b837d03 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.336512 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1364/2700] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:16.347217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfd524-0d36-49cb-92bf-293ec4785eda None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.348964 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-21dfd524-0d36-49cb-92bf-293ec4785eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.356156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1337/2701] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:16 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:16.360363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5025b-2af8-41be-86f2-796b2b837d03 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.444571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfd524-0d36-49cb-92bf-293ec4785eda None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.447253 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5025b-2af8-41be-86f2-796b2b837d03 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.452435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff3ec4-1998-494a-8c2f-f0fe3e7803b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.453150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff3ec4-1998-494a-8c2f-f0fe3e7803b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.453472 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff3ec4-1998-494a-8c2f-f0fe3e7803b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.453725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff3ec4-1998-494a-8c2f-f0fe3e7803b6 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.454559 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff3ec4-1998-494a-8c2f-f0fe3e7803b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.456383 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff3ec4-1998-494a-8c2f-f0fe3e7803b6 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.478982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1338/2702] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:16 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:16.479841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff3ec4-1998-494a-8c2f-f0fe3e7803b6 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.482483 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-11ff3ec4-1998-494a-8c2f-f0fe3e7803b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.499030 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e88eac-11a2-49f0-b4ec-d0bf2be64d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.499685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e88eac-11a2-49f0-b4ec-d0bf2be64d08 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.499970 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e88eac-11a2-49f0-b4ec-d0bf2be64d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.500250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e88eac-11a2-49f0-b4ec-d0bf2be64d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.509661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e88eac-11a2-49f0-b4ec-d0bf2be64d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.511383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e88eac-11a2-49f0-b4ec-d0bf2be64d08 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.517012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:16.521729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1365/2703] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:16.523025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e88eac-11a2-49f0-b4ec-d0bf2be64d08 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.533905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e88eac-11a2-49f0-b4ec-d0bf2be64d08 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.538699 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ad40-ab95-4a9e-8dec-d09c54464883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.539363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ad40-ab95-4a9e-8dec-d09c54464883 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.539648 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ad40-ab95-4a9e-8dec-d09c54464883 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.539975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ad40-ab95-4a9e-8dec-d09c54464883 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.540827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ad40-ab95-4a9e-8dec-d09c54464883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.542464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ad40-ab95-4a9e-8dec-d09c54464883 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.545211 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff3ec4-1998-494a-8c2f-f0fe3e7803b6 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.550209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c063a-fd4d-4bd6-8f0e-7daac86936ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.551638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c063a-fd4d-4bd6-8f0e-7daac86936ce service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.552162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c063a-fd4d-4bd6-8f0e-7daac86936ce service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.552689 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c063a-fd4d-4bd6-8f0e-7daac86936ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.552820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1366/2704] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:16.554964 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-756c063a-fd4d-4bd6-8f0e-7daac86936ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.556839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ad40-ab95-4a9e-8dec-d09c54464883 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.567593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ad40-ab95-4a9e-8dec-d09c54464883 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.570583 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c063a-fd4d-4bd6-8f0e-7daac86936ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.572688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c063a-fd4d-4bd6-8f0e-7daac86936ce service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.574550 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ac65-81b2-4871-a825-824742726b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.575656 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ac65-81b2-4871-a825-824742726b98 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.575851 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ac65-81b2-4871-a825-824742726b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.576133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ac65-81b2-4871-a825-824742726b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.576913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ac65-81b2-4871-a825-824742726b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.578581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ac65-81b2-4871-a825-824742726b98 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.585007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1339/2705] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:16.592502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1367/2706] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:56:16.601753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ac65-81b2-4871-a825-824742726b98 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.614409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ac65-81b2-4871-a825-824742726b98 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.620123 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628eb5f-bbf6-4818-ab4b-02433e62c378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.621011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628eb5f-bbf6-4818-ab4b-02433e62c378 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.621204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628eb5f-bbf6-4818-ab4b-02433e62c378 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.621539 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628eb5f-bbf6-4818-ab4b-02433e62c378 admin admin] PATH_INFO: `/v3/projects/111dd3ec10d045868bcfb448ae4a148c/users/0ea9561b2d614c3ab24bcc9d465ac26d/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.643023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628eb5f-bbf6-4818-ab4b-02433e62c378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111dd3ec10d045868bcfb448ae4a148c, user_id=0ea9561b2d614c3ab24bcc9d465ac26d, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.645770 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628eb5f-bbf6-4818-ab4b-02433e62c378 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.672472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1368/2707] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:16 2022] PUT /identity/v3/projects/111dd3ec10d045868bcfb448ae4a148c/users/0ea9561b2d614c3ab24bcc9d465ac26d/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:16.675534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af6c36-a738-4778-8932-10826f9fb108 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.675990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af6c36-a738-4778-8932-10826f9fb108 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.676098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af6c36-a738-4778-8932-10826f9fb108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.710737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-39af6c36-a738-4778-8932-10826f9fb108 None None] MFA Rules not processed for user `0ea9561b2d614c3ab24bcc9d465ac26d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:16.712064 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-39af6c36-a738-4778-8932-10826f9fb108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.795188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1340/2708] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:16.804324 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628eb5f-bbf6-4818-ab4b-02433e62c378 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.867780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af6c36-a738-4778-8932-10826f9fb108 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.870243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-39af6c36-a738-4778-8932-10826f9fb108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:16.888282 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628eb5f-bbf6-4818-ab4b-02433e62c378 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:16.894011 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cb82d-f012-4ce1-84ea-b3acefd86184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:16.894815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cb82d-f012-4ce1-84ea-b3acefd86184 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:16.895230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cb82d-f012-4ce1-84ea-b3acefd86184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:16.895588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836cb82d-f012-4ce1-84ea-b3acefd86184 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:16.896398 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836cb82d-f012-4ce1-84ea-b3acefd86184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:16.898503 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836cb82d-f012-4ce1-84ea-b3acefd86184 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:16.909937 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:16.937251 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1369/2709] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:16 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:16.940426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cb82d-f012-4ce1-84ea-b3acefd86184 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:16.946426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af6c36-a738-4778-8932-10826f9fb108 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.011244 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836cb82d-f012-4ce1-84ea-b3acefd86184 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.016271 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248f75a-1691-435f-9506-fca92f5b0ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.017365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248f75a-1691-435f-9506-fca92f5b0ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.018089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248f75a-1691-435f-9506-fca92f5b0ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.018538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248f75a-1691-435f-9506-fca92f5b0ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.019305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248f75a-1691-435f-9506-fca92f5b0ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.021239 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.021597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248f75a-1691-435f-9506-fca92f5b0ab2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.022109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.022391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.022666 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] PATH_INFO: `/v3/projects/9c6c8c69f9cb4c4ca5b4e71340689bad` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.023909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c6c8c69f9cb4c4ca5b4e71340689bad)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.027189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.047052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=100 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:17.050404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1370/2710] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:16 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:17.054775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248f75a-1691-435f-9506-fca92f5b0ab2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.069714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248f75a-1691-435f-9506-fca92f5b0ab2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.084807 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6c8c69f9cb4c4ca5b4e71340689bad', 'request_id': 'req-65ca74e5-b672-4cf7-9074-cebcdf637a34'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:17.095351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6c8c69f9cb4c4ca5b4e71340689bad', 'request_id': 'req-65ca74e5-b672-4cf7-9074-cebcdf637a34'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:17.104214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6c8c69f9cb4c4ca5b4e71340689bad', 'request_id': 'req-65ca74e5-b672-4cf7-9074-cebcdf637a34'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:17.115659 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] The token cache is being invalidate because project 9c6c8c69f9cb4c4ca5b4e71340689bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:17.116161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:17.117000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:17.119204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1341/2711] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:16 2022] DELETE /identity/v3/projects/9c6c8c69f9cb4c4ca5b4e71340689bad => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:17.136672 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be07cf-9858-4b08-8eab-78b7490bd8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.137705 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be07cf-9858-4b08-8eab-78b7490bd8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.138342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be07cf-9858-4b08-8eab-78b7490bd8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.138787 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be07cf-9858-4b08-8eab-78b7490bd8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.139780 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be07cf-9858-4b08-8eab-78b7490bd8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.141802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be07cf-9858-4b08-8eab-78b7490bd8b2 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.159005 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1371/2712] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:17.160642 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.163309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:17.178705 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca74e5-b672-4cf7-9074-cebcdf637a34 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.184872 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76daa2-6c27-47c4-90b3-8afc51c77021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.185734 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76daa2-6c27-47c4-90b3-8afc51c77021 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.186123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76daa2-6c27-47c4-90b3-8afc51c77021 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.186535 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76daa2-6c27-47c4-90b3-8afc51c77021 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.187425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76daa2-6c27-47c4-90b3-8afc51c77021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.189280 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76daa2-6c27-47c4-90b3-8afc51c77021 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.205361 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1342/2713] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:17.206824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be07cf-9858-4b08-8eab-78b7490bd8b2 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.218455 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be07cf-9858-4b08-8eab-78b7490bd8b2 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.223570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddbb1c-7648-4c12-b958-d67aa8c39789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.224686 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddbb1c-7648-4c12-b958-d67aa8c39789 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.225072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddbb1c-7648-4c12-b958-d67aa8c39789 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.225479 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddbb1c-7648-4c12-b958-d67aa8c39789 admin admin] PATH_INFO: `/v3/projects/cbd479eee2cd4b04a5875fe7e9558a11/users/8523ce5533764f44b692174b8f6fdc5b/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.245854 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddbb1c-7648-4c12-b958-d67aa8c39789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd479eee2cd4b04a5875fe7e9558a11, user_id=8523ce5533764f44b692174b8f6fdc5b, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.249061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddbb1c-7648-4c12-b958-d67aa8c39789 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.274013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1372/2714] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:17 2022] PUT /identity/v3/projects/cbd479eee2cd4b04a5875fe7e9558a11/users/8523ce5533764f44b692174b8f6fdc5b/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:17.281307 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89972bf-1d75-4821-aa17-86f5b43cda42 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.281976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89972bf-1d75-4821-aa17-86f5b43cda42 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.282494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89972bf-1d75-4821-aa17-86f5b43cda42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.323948 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f89972bf-1d75-4821-aa17-86f5b43cda42 None None] MFA Rules not processed for user `8523ce5533764f44b692174b8f6fdc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:17.325554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f89972bf-1d75-4821-aa17-86f5b43cda42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:17.406199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1343/2715] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:17.410696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddbb1c-7648-4c12-b958-d67aa8c39789 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.465092 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:17.477226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddbb1c-7648-4c12-b958-d67aa8c39789 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.482902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f24cb6-029c-4414-8c47-65b878d55b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.483597 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f24cb6-029c-4414-8c47-65b878d55b87 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.483885 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f24cb6-029c-4414-8c47-65b878d55b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.484168 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f24cb6-029c-4414-8c47-65b878d55b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.484912 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f24cb6-029c-4414-8c47-65b878d55b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.486823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f24cb6-029c-4414-8c47-65b878d55b87 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.514064 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1373/2716] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:17 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:17.518697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89972bf-1d75-4821-aa17-86f5b43cda42 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.529787 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89972bf-1d75-4821-aa17-86f5b43cda42 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.586875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628ff8f-55e7-419a-8b52-40906047c221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.587583 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628ff8f-55e7-419a-8b52-40906047c221 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.588841 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628ff8f-55e7-419a-8b52-40906047c221 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.589197 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628ff8f-55e7-419a-8b52-40906047c221 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.590097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628ff8f-55e7-419a-8b52-40906047c221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.592192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628ff8f-55e7-419a-8b52-40906047c221 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.612772 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1344/2717] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:17 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:17.616638 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f24cb6-029c-4414-8c47-65b878d55b87 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.627206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f24cb6-029c-4414-8c47-65b878d55b87 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.631974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57b53b-e68a-46ad-ac00-88126c6b18f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.633002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57b53b-e68a-46ad-ac00-88126c6b18f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.633292 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57b53b-e68a-46ad-ac00-88126c6b18f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.633566 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57b53b-e68a-46ad-ac00-88126c6b18f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.634287 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57b53b-e68a-46ad-ac00-88126c6b18f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.636559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57b53b-e68a-46ad-ac00-88126c6b18f4 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.647380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1374/2718] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:17.651640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628ff8f-55e7-419a-8b52-40906047c221 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.662095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628ff8f-55e7-419a-8b52-40906047c221 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.667962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb576791-ffd6-4ab2-b068-cf3fc25763a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.669074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb576791-ffd6-4ab2-b068-cf3fc25763a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.669212 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb576791-ffd6-4ab2-b068-cf3fc25763a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.669438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb576791-ffd6-4ab2-b068-cf3fc25763a3 admin admin] PATH_INFO: `/v3/projects/d0415ae6b1f34ff1bbe4b75725d66ced/users/6de1009b88fd4969bd63da24143c4cdf/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.678059 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:17.685990 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb576791-ffd6-4ab2-b068-cf3fc25763a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0415ae6b1f34ff1bbe4b75725d66ced, user_id=6de1009b88fd4969bd63da24143c4cdf, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.690133 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb576791-ffd6-4ab2-b068-cf3fc25763a3 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.703794 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1345/2719] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:17 2022] PUT /identity/v3/projects/d0415ae6b1f34ff1bbe4b75725d66ced/users/6de1009b88fd4969bd63da24143c4cdf/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:17.708033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57b53b-e68a-46ad-ac00-88126c6b18f4 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.719086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57b53b-e68a-46ad-ac00-88126c6b18f4 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.795201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72149e92-0fec-4e19-8e52-b35955f9499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.796069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72149e92-0fec-4e19-8e52-b35955f9499a admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.796479 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72149e92-0fec-4e19-8e52-b35955f9499a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.796882 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72149e92-0fec-4e19-8e52-b35955f9499a admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.797698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72149e92-0fec-4e19-8e52-b35955f9499a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.799357 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72149e92-0fec-4e19-8e52-b35955f9499a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.810327 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1375/2720] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:17.814421 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb576791-ffd6-4ab2-b068-cf3fc25763a3 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.825548 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb576791-ffd6-4ab2-b068-cf3fc25763a3 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:17.830426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d09058-7383-421e-8707-b22031f8278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:17.831317 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d09058-7383-421e-8707-b22031f8278b admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.831613 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d09058-7383-421e-8707-b22031f8278b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.831893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d09058-7383-421e-8707-b22031f8278b admin admin] PATH_INFO: `/v3/projects/d0415ae6b1f34ff1bbe4b75725d66ced/users/6de1009b88fd4969bd63da24143c4cdf/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.834001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d09058-7383-421e-8707-b22031f8278b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0415ae6b1f34ff1bbe4b75725d66ced, user_id=6de1009b88fd4969bd63da24143c4cdf, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:17.836692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d09058-7383-421e-8707-b22031f8278b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:17.849098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1346/2721] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:17 2022] PUT /identity/v3/projects/d0415ae6b1f34ff1bbe4b75725d66ced/users/6de1009b88fd4969bd63da24143c4cdf/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:17.860905 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d253f35-e307-48bf-859b-446f0979c238 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:17.861218 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d253f35-e307-48bf-859b-446f0979c238 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:17.861500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d253f35-e307-48bf-859b-446f0979c238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:17.891396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9d253f35-e307-48bf-859b-446f0979c238 None None] MFA Rules not processed for user `6de1009b88fd4969bd63da24143c4cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:17.892317 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9d253f35-e307-48bf-859b-446f0979c238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:17.980145 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1376/2722] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:17.991879 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d09058-7383-421e-8707-b22031f8278b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:17.994442 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-35d09058-7383-421e-8707-b22031f8278b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:18.066878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d09058-7383-421e-8707-b22031f8278b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:18.070671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a251d-bf65-4d9f-bfca-49ab0807aa49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:18.071323 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a251d-bf65-4d9f-bfca-49ab0807aa49 service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:18.071606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a251d-bf65-4d9f-bfca-49ab0807aa49 service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:18.071885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a251d-bf65-4d9f-bfca-49ab0807aa49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:18.074792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a30a251d-bf65-4d9f-bfca-49ab0807aa49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:18.143644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a251d-bf65-4d9f-bfca-49ab0807aa49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:18.145067 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a251d-bf65-4d9f-bfca-49ab0807aa49 service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:18.155098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1347/2723] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:19.692793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d253f35-e307-48bf-859b-446f0979c238 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:19.703437 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d253f35-e307-48bf-859b-446f0979c238 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:19.707217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b47a7-5471-4833-b0ba-a1beccc81332 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:19.707995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b47a7-5471-4833-b0ba-a1beccc81332 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:19.708149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b47a7-5471-4833-b0ba-a1beccc81332 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:19.708426 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b47a7-5471-4833-b0ba-a1beccc81332 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:19.756982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:19.775929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b47a7-5471-4833-b0ba-a1beccc81332 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:19.778141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b47a7-5471-4833-b0ba-a1beccc81332 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:19.789335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1377/2724] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:20.522355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a251d-bf65-4d9f-bfca-49ab0807aa49 service cinder] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:20.531814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a251d-bf65-4d9f-bfca-49ab0807aa49 service cinder] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:20.535543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d02c8-3a71-4d62-a7e9-804800474c0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:20.536270 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d02c8-3a71-4d62-a7e9-804800474c0a service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:20.536647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d02c8-3a71-4d62-a7e9-804800474c0a service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:20.536991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d02c8-3a71-4d62-a7e9-804800474c0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:20.546746 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d02c8-3a71-4d62-a7e9-804800474c0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:20.548240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d02c8-3a71-4d62-a7e9-804800474c0a service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:20.558140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1348/2725] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:20 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:21.329297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b47a7-5471-4833-b0ba-a1beccc81332 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:21.330823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-648b47a7-5471-4833-b0ba-a1beccc81332 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:21.403209 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b47a7-5471-4833-b0ba-a1beccc81332 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:21.410352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:21.411867 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:21.412376 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:21.412940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] PATH_INFO: `/v3/users/8204dee5ba6e4049b0bb25f0389cb4e8` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:21.414230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8204dee5ba6e4049b0bb25f0389cb4e8)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:21.416337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:21.465991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8204dee5ba6e4049b0bb25f0389cb4e8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:21.478115 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8204dee5ba6e4049b0bb25f0389cb4e8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:21.479249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8204dee5ba6e4049b0bb25f0389cb4e8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:21.479863 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8204dee5ba6e4049b0bb25f0389cb4e8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:21.487559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8204dee5ba6e4049b0bb25f0389cb4e8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:21.503305 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1378/2726] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:21 2022] DELETE /identity/v3/users/8204dee5ba6e4049b0bb25f0389cb4e8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:21.510508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ac13d-6cab-4c5f-9518-3a440a5048c4 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:21.511032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ac13d-6cab-4c5f-9518-3a440a5048c4 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:21.511485 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ac13d-6cab-4c5f-9518-3a440a5048c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:21.534306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-7d0ac13d-6cab-4c5f-9518-3a440a5048c4 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:21.535592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7d0ac13d-6cab-4c5f-9518-3a440a5048c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:21.606462 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1349/2727] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:21.614801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:21.616274 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:21.682822 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b666d98-fbb8-44d4-a5f3-ae26374d1618 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:21.691099 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1f76d-883c-4ff1-8c3d-7d2ed73e5b69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:21.691837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1f76d-883c-4ff1-8c3d-7d2ed73e5b69 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:21.692192 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1f76d-883c-4ff1-8c3d-7d2ed73e5b69 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:21.692537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1f76d-883c-4ff1-8c3d-7d2ed73e5b69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:21.704277 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1f76d-883c-4ff1-8c3d-7d2ed73e5b69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:21.706044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1f76d-883c-4ff1-8c3d-7d2ed73e5b69 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:21.717375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1379/2728] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:22.112043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ac13d-6cab-4c5f-9518-3a440a5048c4 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:22.121926 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7d0ac13d-6cab-4c5f-9518-3a440a5048c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:22.143547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ac13d-6cab-4c5f-9518-3a440a5048c4 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:22.152444 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:22.153761 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.154288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.155434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] PATH_INFO: `/v3/projects/46f447736b9243bf96804fdbe7e194da` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.157267 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f447736b9243bf96804fdbe7e194da)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:22.162261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:22.175516 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:22.217751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f447736b9243bf96804fdbe7e194da', 'request_id': 'req-12e5e43d-0bd0-4e2a-b00a-04e375b80119'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:22.225538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f447736b9243bf96804fdbe7e194da', 'request_id': 'req-12e5e43d-0bd0-4e2a-b00a-04e375b80119'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:22.237402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f447736b9243bf96804fdbe7e194da', 'request_id': 'req-12e5e43d-0bd0-4e2a-b00a-04e375b80119'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:22.248838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] The token cache is being invalidate because project 46f447736b9243bf96804fdbe7e194da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:22.249062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:22.249806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-12e5e43d-0bd0-4e2a-b00a-04e375b80119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:22.251347 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1350/2729] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:22 2022] DELETE /identity/v3/projects/46f447736b9243bf96804fdbe7e194da => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:22.256225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711bbba-8b4a-4cf6-bb2a-580fdfd74f7d None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.256538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711bbba-8b4a-4cf6-bb2a-580fdfd74f7d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.256846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711bbba-8b4a-4cf6-bb2a-580fdfd74f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.269839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:22.277054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-d711bbba-8b4a-4cf6-bb2a-580fdfd74f7d None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:22.278918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d711bbba-8b4a-4cf6-bb2a-580fdfd74f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:22.346791 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1380/2730] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:22.354937 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e9482-8229-49af-adfc-6376c38c1953 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.355231 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e9482-8229-49af-adfc-6376c38c1953 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.355518 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e9482-8229-49af-adfc-6376c38c1953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.374045 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-990e9482-8229-49af-adfc-6376c38c1953 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:22.375261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-990e9482-8229-49af-adfc-6376c38c1953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:22.383292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1351/2731] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:22.387603 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711bbba-8b4a-4cf6-bb2a-580fdfd74f7d None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:22.397714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711bbba-8b4a-4cf6-bb2a-580fdfd74f7d None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:22.403404 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7f66c-baa7-4ac4-bffa-f79793520713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:22.404260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7f66c-baa7-4ac4-bffa-f79793520713 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.404801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7f66c-baa7-4ac4-bffa-f79793520713 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.405268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7f66c-baa7-4ac4-bffa-f79793520713 admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.406025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7f66c-baa7-4ac4-bffa-f79793520713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:22.406522 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7f66c-baa7-4ac4-bffa-f79793520713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:22.408659 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7f66c-baa7-4ac4-bffa-f79793520713 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:22.424658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1381/2732] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:22.425875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e9482-8229-49af-adfc-6376c38c1953 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:22.435591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e9482-8229-49af-adfc-6376c38c1953 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:22.441102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6602bb5-c0ed-4872-87d9-5bc54f8e4e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:22.442204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6602bb5-c0ed-4872-87d9-5bc54f8e4e4e admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.442701 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6602bb5-c0ed-4872-87d9-5bc54f8e4e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.443228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6602bb5-c0ed-4872-87d9-5bc54f8e4e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.444537 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6602bb5-c0ed-4872-87d9-5bc54f8e4e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:22.446526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6602bb5-c0ed-4872-87d9-5bc54f8e4e4e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:22.481114 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1352/2733] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:22 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:22.483054 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7f66c-baa7-4ac4-bffa-f79793520713 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:22.493698 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7f66c-baa7-4ac4-bffa-f79793520713 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:22.560242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2b945-b043-475c-867c-704332bd95f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:22.561049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2b945-b043-475c-867c-704332bd95f8 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.561338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2b945-b043-475c-867c-704332bd95f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.561611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2b945-b043-475c-867c-704332bd95f8 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.562511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2b945-b043-475c-867c-704332bd95f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:22.564446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2b945-b043-475c-867c-704332bd95f8 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:22.588753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1382/2734] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:22 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:22.592525 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6602bb5-c0ed-4872-87d9-5bc54f8e4e4e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:22.604799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6602bb5-c0ed-4872-87d9-5bc54f8e4e4e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:22.610520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098112e-cf7b-438f-9f87-e59dcdf87216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:22.611540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098112e-cf7b-438f-9f87-e59dcdf87216 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.611540 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098112e-cf7b-438f-9f87-e59dcdf87216 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.611797 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098112e-cf7b-438f-9f87-e59dcdf87216 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.612547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098112e-cf7b-438f-9f87-e59dcdf87216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:22.614237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098112e-cf7b-438f-9f87-e59dcdf87216 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:22.629565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1353/2735] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:22.631033 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2b945-b043-475c-867c-704332bd95f8 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:22.642295 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2b945-b043-475c-867c-704332bd95f8 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:22.647516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adaeaf-9543-4c47-bbba-e2cbf33d5f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:22.648178 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adaeaf-9543-4c47-bbba-e2cbf33d5f58 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.648487 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adaeaf-9543-4c47-bbba-e2cbf33d5f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.648800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adaeaf-9543-4c47-bbba-e2cbf33d5f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.649543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adaeaf-9543-4c47-bbba-e2cbf33d5f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:22.651549 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adaeaf-9543-4c47-bbba-e2cbf33d5f58 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:22.662066 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1383/2736] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:56:22.666839 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098112e-cf7b-438f-9f87-e59dcdf87216 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:22.677938 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098112e-cf7b-438f-9f87-e59dcdf87216 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:22.683923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f95da6-d014-4c52-844b-7fcf8e4ff1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:22.684866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f95da6-d014-4c52-844b-7fcf8e4ff1de admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.685150 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f95da6-d014-4c52-844b-7fcf8e4ff1de admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.685453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f95da6-d014-4c52-844b-7fcf8e4ff1de admin admin] PATH_INFO: `/v3/projects/40c3d04f5ceb4b57ae8e560988388c60/users/ba77a2d06c97434e93ea3aa85fd74b90/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.701524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f95da6-d014-4c52-844b-7fcf8e4ff1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c3d04f5ceb4b57ae8e560988388c60, user_id=ba77a2d06c97434e93ea3aa85fd74b90, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:22.704417 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f95da6-d014-4c52-844b-7fcf8e4ff1de admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:22.719529 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1354/2737] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:22 2022] PUT /identity/v3/projects/40c3d04f5ceb4b57ae8e560988388c60/users/ba77a2d06c97434e93ea3aa85fd74b90/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:22.731403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da5a7a-6f28-413c-9f2e-b024db69dd5b None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.731697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da5a7a-6f28-413c-9f2e-b024db69dd5b None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.732097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da5a7a-6f28-413c-9f2e-b024db69dd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.787281 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-84da5a7a-6f28-413c-9f2e-b024db69dd5b None None] MFA Rules not processed for user `ba77a2d06c97434e93ea3aa85fd74b90`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:22.789824 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-84da5a7a-6f28-413c-9f2e-b024db69dd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:22.872717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1384/2738] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:22.876444 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f95da6-d014-4c52-844b-7fcf8e4ff1de admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:22.962176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f95da6-d014-4c52-844b-7fcf8e4ff1de admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:22.968569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6196f-6aba-4747-9664-7d0d54c9854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:22.969563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6196f-6aba-4747-9664-7d0d54c9854a admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:22.970030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6196f-6aba-4747-9664-7d0d54c9854a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:22.970598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6196f-6aba-4747-9664-7d0d54c9854a admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:22.971627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6196f-6aba-4747-9664-7d0d54c9854a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:22.973773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6196f-6aba-4747-9664-7d0d54c9854a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:23.012704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1355/2739] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:22 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:23.014318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da5a7a-6f28-413c-9f2e-b024db69dd5b None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:23.021090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:23.027742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da5a7a-6f28-413c-9f2e-b024db69dd5b None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:23.135216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a232a84-92a2-4a1a-8074-428098ec2aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:23.136434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a232a84-92a2-4a1a-8074-428098ec2aef admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:23.136984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a232a84-92a2-4a1a-8074-428098ec2aef admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:23.137461 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a232a84-92a2-4a1a-8074-428098ec2aef admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:23.138511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a232a84-92a2-4a1a-8074-428098ec2aef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:23.140552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a232a84-92a2-4a1a-8074-428098ec2aef admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:23.169719 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1385/2740] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:23 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:23.173289 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6196f-6aba-4747-9664-7d0d54c9854a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:23.186363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6196f-6aba-4747-9664-7d0d54c9854a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:23.195230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230520ab-8b1c-45fd-9b06-390a0f80cbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:23.197429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230520ab-8b1c-45fd-9b06-390a0f80cbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:23.198074 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230520ab-8b1c-45fd-9b06-390a0f80cbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:23.198773 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230520ab-8b1c-45fd-9b06-390a0f80cbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:23.200266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230520ab-8b1c-45fd-9b06-390a0f80cbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:23.203834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230520ab-8b1c-45fd-9b06-390a0f80cbb8 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:23.223501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1356/2741] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:23.227379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a232a84-92a2-4a1a-8074-428098ec2aef admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:23.241800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a232a84-92a2-4a1a-8074-428098ec2aef admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:23.266554 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a3790-7f7a-47b1-9c9d-90ca36cd596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:23.267200 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a3790-7f7a-47b1-9c9d-90ca36cd596e admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:23.267802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a3790-7f7a-47b1-9c9d-90ca36cd596e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:23.268162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a3790-7f7a-47b1-9c9d-90ca36cd596e admin admin] PATH_INFO: `/v3/projects/83077bece7854caea7b4791f9b52b525/users/8856cb6a38ed41bca9662dfbcc2c727b/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:23.298199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a3790-7f7a-47b1-9c9d-90ca36cd596e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83077bece7854caea7b4791f9b52b525, user_id=8856cb6a38ed41bca9662dfbcc2c727b, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:23.301155 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a3790-7f7a-47b1-9c9d-90ca36cd596e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:23.318601 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1386/2742] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:23 2022] PUT /identity/v3/projects/83077bece7854caea7b4791f9b52b525/users/8856cb6a38ed41bca9662dfbcc2c727b/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:23.323156 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230520ab-8b1c-45fd-9b06-390a0f80cbb8 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:23.333580 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230520ab-8b1c-45fd-9b06-390a0f80cbb8 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:23.339108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:23.369380 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a3790-7f7a-47b1-9c9d-90ca36cd596e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:23.371730 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e80a3790-7f7a-47b1-9c9d-90ca36cd596e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:23.432202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51bd57-3be2-495f-bd9a-18dd9e146d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:23.432947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51bd57-3be2-495f-bd9a-18dd9e146d03 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:23.433227 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51bd57-3be2-495f-bd9a-18dd9e146d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:23.433538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51bd57-3be2-495f-bd9a-18dd9e146d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:23.434477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51bd57-3be2-495f-bd9a-18dd9e146d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:23.435924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51bd57-3be2-495f-bd9a-18dd9e146d03 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:23.454081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1357/2743] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:23.454994 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51bd57-3be2-495f-bd9a-18dd9e146d03 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:23.469441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51bd57-3be2-495f-bd9a-18dd9e146d03 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:23.476000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d087c5-7a96-4a5f-9ef5-5d50fc918e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:23.477040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d087c5-7a96-4a5f-9ef5-5d50fc918e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:23.477447 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d087c5-7a96-4a5f-9ef5-5d50fc918e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:23.477795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d087c5-7a96-4a5f-9ef5-5d50fc918e45 admin admin] PATH_INFO: `/v3/projects/83077bece7854caea7b4791f9b52b525/users/8856cb6a38ed41bca9662dfbcc2c727b/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:23.483277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d087c5-7a96-4a5f-9ef5-5d50fc918e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83077bece7854caea7b4791f9b52b525, user_id=8856cb6a38ed41bca9662dfbcc2c727b, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:23.486411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d087c5-7a96-4a5f-9ef5-5d50fc918e45 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:23.487384 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a3790-7f7a-47b1-9c9d-90ca36cd596e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:23.494545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989ce2b-d7a7-4735-9582-80d613bbd768 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:23.495477 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989ce2b-d7a7-4735-9582-80d613bbd768 service placement] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:23.495792 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989ce2b-d7a7-4735-9582-80d613bbd768 service placement] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:23.496399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989ce2b-d7a7-4735-9582-80d613bbd768 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:23.501569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-f989ce2b-d7a7-4735-9582-80d613bbd768 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:23.509869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1358/2744] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:23 2022] PUT /identity/v3/projects/83077bece7854caea7b4791f9b52b525/users/8856cb6a38ed41bca9662dfbcc2c727b/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:23.509869 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d087c5-7a96-4a5f-9ef5-5d50fc918e45 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:23.511492 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b6d087c5-7a96-4a5f-9ef5-5d50fc918e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:23.524767 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989ce2b-d7a7-4735-9582-80d613bbd768 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:23.527310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f989ce2b-d7a7-4735-9582-80d613bbd768 service placement] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:23.540105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1387/2745] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:23.541561 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31072ed8-e90d-4d7d-88f0-cf82c44f81a9 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:23.541856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31072ed8-e90d-4d7d-88f0-cf82c44f81a9 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:23.542137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31072ed8-e90d-4d7d-88f0-cf82c44f81a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:23.588712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-31072ed8-e90d-4d7d-88f0-cf82c44f81a9 None None] MFA Rules not processed for user `8856cb6a38ed41bca9662dfbcc2c727b`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:23.590345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-31072ed8-e90d-4d7d-88f0-cf82c44f81a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:23.612663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d087c5-7a96-4a5f-9ef5-5d50fc918e45 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:23.617548 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:23.618572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:23.618893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:23.619175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] PATH_INFO: `/v3/users/7faba3540b1b4ef5a533ab0ff8e1cca3` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:23.620590 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7faba3540b1b4ef5a533ab0ff8e1cca3)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:23.622523 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:23.647035 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:23.673789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7faba3540b1b4ef5a533ab0ff8e1cca3'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:23.674544 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1388/2746] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:23.687299 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31072ed8-e90d-4d7d-88f0-cf82c44f81a9 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:23.689315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-31072ed8-e90d-4d7d-88f0-cf82c44f81a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:23.696085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7faba3540b1b4ef5a533ab0ff8e1cca3'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:23.706038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7faba3540b1b4ef5a533ab0ff8e1cca3'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:23.706711 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7faba3540b1b4ef5a533ab0ff8e1cca3'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:23.714733 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7faba3540b1b4ef5a533ab0ff8e1cca3'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:23.724758 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1359/2747] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:23 2022] DELETE /identity/v3/users/7faba3540b1b4ef5a533ab0ff8e1cca3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:23.762731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31072ed8-e90d-4d7d-88f0-cf82c44f81a9 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:23.820571 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:23.821260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:23.821545 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:23.822489 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:23.824454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:23.889940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:23.891505 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:23.901761 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1389/2748] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:23.912327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:23.913853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:23.941976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:24.000637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cb084-1ffa-47da-bfb6-1fb7172f5ef9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:24.001995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:24.003558 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:24.005898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:24.006819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:24.007181 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:24.007519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] PATH_INFO: `/v3/projects/e318cfdc9de345148828c09515cecdc1` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:24.008781 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e318cfdc9de345148828c09515cecdc1)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:24.011598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:24.015122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc3c54-f6c2-4415-ad7a-b282e8c2776a service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:24.019105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9148a-d722-44ae-bfeb-158dc09ae314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:24.019747 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9148a-d722-44ae-bfeb-158dc09ae314 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:24.020037 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9148a-d722-44ae-bfeb-158dc09ae314 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:24.020319 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9148a-d722-44ae-bfeb-158dc09ae314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:24.022170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-bda9148a-d722-44ae-bfeb-158dc09ae314 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:24.061801 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e318cfdc9de345148828c09515cecdc1', 'request_id': 'req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.067939 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e318cfdc9de345148828c09515cecdc1', 'request_id': 'req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.072022 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=100 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:24.076338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e318cfdc9de345148828c09515cecdc1', 'request_id': 'req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.087584 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9148a-d722-44ae-bfeb-158dc09ae314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:24.087722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] The token cache is being invalidate because project e318cfdc9de345148828c09515cecdc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:24.087944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.088597 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.089089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9148a-d722-44ae-bfeb-158dc09ae314 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:24.090357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1360/2749] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:23 2022] DELETE /identity/v3/projects/e318cfdc9de345148828c09515cecdc1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:24.095777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:24.097618 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:24.103488 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1390/2750] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:24.175071 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7013dd-4b4d-4026-b9c3-af65dc0d921a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:24.180100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:24.181165 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:24.181514 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:24.181860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] PATH_INFO: `/v3/users/e71329a343964a4d9a53ca144652da58` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:24.183140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71329a343964a4d9a53ca144652da58)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:24.185507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:24.245238 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:24.248704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71329a343964a4d9a53ca144652da58'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.264550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71329a343964a4d9a53ca144652da58'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.273168 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71329a343964a4d9a53ca144652da58'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.274053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71329a343964a4d9a53ca144652da58'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.282328 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71329a343964a4d9a53ca144652da58'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.288434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1361/2751] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:24 2022] DELETE /identity/v3/users/e71329a343964a4d9a53ca144652da58 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:24.484754 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9148a-d722-44ae-bfeb-158dc09ae314 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:24.497144 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9148a-d722-44ae-bfeb-158dc09ae314 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:24.558442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:24.559304 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:24.559591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:24.559871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] PATH_INFO: `/v3/projects/edff0e7d198c4c93b8bb38caffc71217` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:24.561091 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=edff0e7d198c4c93b8bb38caffc71217)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:24.563009 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:24.609273 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edff0e7d198c4c93b8bb38caffc71217', 'request_id': 'req-b2151a16-064b-4776-a24b-d40c982064eb'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.616338 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edff0e7d198c4c93b8bb38caffc71217', 'request_id': 'req-b2151a16-064b-4776-a24b-d40c982064eb'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.622834 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edff0e7d198c4c93b8bb38caffc71217', 'request_id': 'req-b2151a16-064b-4776-a24b-d40c982064eb'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.633583 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] The token cache is being invalidate because project edff0e7d198c4c93b8bb38caffc71217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:24.633869 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.634590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-b2151a16-064b-4776-a24b-d40c982064eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.636206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1391/2752] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:24 2022] DELETE /identity/v3/projects/edff0e7d198c4c93b8bb38caffc71217 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:24.691467 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:24.693652 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:24.715640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1eacc-a669-41ec-8ba5-bf9c4ea5ec85 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:24.786765 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:24.787660 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:24.787944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:24.788224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] PATH_INFO: `/v3/users/0ea9561b2d614c3ab24bcc9d465ac26d` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:24.789512 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea9561b2d614c3ab24bcc9d465ac26d)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:24.792324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:24.849234 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea9561b2d614c3ab24bcc9d465ac26d'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.863450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea9561b2d614c3ab24bcc9d465ac26d'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.874604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea9561b2d614c3ab24bcc9d465ac26d'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.875233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea9561b2d614c3ab24bcc9d465ac26d'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.882729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea9561b2d614c3ab24bcc9d465ac26d'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:24.890202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1362/2753] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:24 2022] DELETE /identity/v3/users/0ea9561b2d614c3ab24bcc9d465ac26d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:24.895210 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c413-d4f8-429e-a9f0-42abd23bea6f None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:24.895499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c413-d4f8-429e-a9f0-42abd23bea6f None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:24.895778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915c413-d4f8-429e-a9f0-42abd23bea6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:24.913596 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3915c413-d4f8-429e-a9f0-42abd23bea6f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:24.914810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3915c413-d4f8-429e-a9f0-42abd23bea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:24.974179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1392/2754] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:24.984255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:24.985707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:25.029040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:25.044601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020617a-7c5b-49ca-bd55-be9fe4c41c86 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:25.048263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a29d3a-be33-4ad8-adb8-e68e8715e5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:25.048945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29d3a-be33-4ad8-adb8-e68e8715e5ca service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:25.049231 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29d3a-be33-4ad8-adb8-e68e8715e5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:25.049507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29d3a-be33-4ad8-adb8-e68e8715e5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:25.058637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a29d3a-be33-4ad8-adb8-e68e8715e5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:25.060072 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a29d3a-be33-4ad8-adb8-e68e8715e5ca service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:25.069470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1363/2755] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:25.351159 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c413-d4f8-429e-a9f0-42abd23bea6f None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:25.352711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3915c413-d4f8-429e-a9f0-42abd23bea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:25.367711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915c413-d4f8-429e-a9f0-42abd23bea6f None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:25.374201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:25.375094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:25.375375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:25.375728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] PATH_INFO: `/v3/projects/111dd3ec10d045868bcfb448ae4a148c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:25.376915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=111dd3ec10d045868bcfb448ae4a148c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:25.378811 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:25.391320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:25.428385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111dd3ec10d045868bcfb448ae4a148c', 'request_id': 'req-e2fdfa22-719a-47aa-982a-debb52e8e8c3'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.437502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111dd3ec10d045868bcfb448ae4a148c', 'request_id': 'req-e2fdfa22-719a-47aa-982a-debb52e8e8c3'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.443660 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111dd3ec10d045868bcfb448ae4a148c', 'request_id': 'req-e2fdfa22-719a-47aa-982a-debb52e8e8c3'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.456462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] The token cache is being invalidate because project 111dd3ec10d045868bcfb448ae4a148c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:25.456795 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.457608 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.460388 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1393/2756] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:25 2022] DELETE /identity/v3/projects/111dd3ec10d045868bcfb448ae4a148c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:25.466134 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a29d3a-be33-4ad8-adb8-e68e8715e5ca service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:25.477450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a29d3a-be33-4ad8-adb8-e68e8715e5ca service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:25.545353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:25.546236 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:25.546520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:25.546799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] PATH_INFO: `/v3/users/8523ce5533764f44b692174b8f6fdc5b` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:25.548007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] RBAC: Authorizing `identity:delete_user(user_id=8523ce5533764f44b692174b8f6fdc5b)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:25.550001 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:25.596499 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8523ce5533764f44b692174b8f6fdc5b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.610961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8523ce5533764f44b692174b8f6fdc5b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.618595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8523ce5533764f44b692174b8f6fdc5b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.622024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8523ce5533764f44b692174b8f6fdc5b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.628821 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8523ce5533764f44b692174b8f6fdc5b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:25.635406 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1364/2757] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:25 2022] DELETE /identity/v3/users/8523ce5533764f44b692174b8f6fdc5b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:26.139855 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:26.141377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:26.158206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdfa22-719a-47aa-982a-debb52e8e8c3 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:26.215504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:26.216367 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:26.216685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:26.216966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] PATH_INFO: `/v3/projects/cbd479eee2cd4b04a5875fe7e9558a11` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:26.218154 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd479eee2cd4b04a5875fe7e9558a11)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:26.220072 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:26.265678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd479eee2cd4b04a5875fe7e9558a11', 'request_id': 'req-3c462d27-8d92-4233-8458-f0d907f6d962'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.273026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd479eee2cd4b04a5875fe7e9558a11', 'request_id': 'req-3c462d27-8d92-4233-8458-f0d907f6d962'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.279574 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd479eee2cd4b04a5875fe7e9558a11', 'request_id': 'req-3c462d27-8d92-4233-8458-f0d907f6d962'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.289716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] The token cache is being invalidate because project cbd479eee2cd4b04a5875fe7e9558a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:26.290001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.291149 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.292711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1394/2758] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:26 2022] DELETE /identity/v3/projects/cbd479eee2cd4b04a5875fe7e9558a11 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:26.296679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:26.298176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:26.312019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82151912-748d-40e6-bd29-3d13d66ebcec admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:26.367241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:26.368105 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:26.369104 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:26.369405 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] PATH_INFO: `/v3/users/6de1009b88fd4969bd63da24143c4cdf` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:26.370865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de1009b88fd4969bd63da24143c4cdf)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:26.372798 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:26.375022 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:26.420255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de1009b88fd4969bd63da24143c4cdf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.432561 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de1009b88fd4969bd63da24143c4cdf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.439100 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de1009b88fd4969bd63da24143c4cdf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.439647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de1009b88fd4969bd63da24143c4cdf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.446541 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de1009b88fd4969bd63da24143c4cdf'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.452567 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1365/2759] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:26 2022] DELETE /identity/v3/users/6de1009b88fd4969bd63da24143c4cdf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:26.700074 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:26.701631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:26.715025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c462d27-8d92-4233-8458-f0d907f6d962 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:26.731411 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:26.733192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:26.750101 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:26.788870 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:26.789887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:26.790254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:26.791134 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] PATH_INFO: `/v3/users/ba77a2d06c97434e93ea3aa85fd74b90` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:26.792861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba77a2d06c97434e93ea3aa85fd74b90)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:26.794910 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:26.827351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295526f8-653b-401b-b4be-008b47ad9410 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:26.833881 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:26.835185 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:26.835661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:26.836135 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] PATH_INFO: `/v3/projects/d0415ae6b1f34ff1bbe4b75725d66ced` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:26.838909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0415ae6b1f34ff1bbe4b75725d66ced)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:26.842521 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:26.843974 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba77a2d06c97434e93ea3aa85fd74b90'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.861468 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba77a2d06c97434e93ea3aa85fd74b90'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.862216 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba77a2d06c97434e93ea3aa85fd74b90'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.862836 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba77a2d06c97434e93ea3aa85fd74b90'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.871971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-4dcc5a90-a3c4-45e5-9a33-d540164e9b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba77a2d06c97434e93ea3aa85fd74b90'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.886623 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1395/2760] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:26 2022] DELETE /identity/v3/users/ba77a2d06c97434e93ea3aa85fd74b90 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:26.890971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47238e92-6230-4793-8791-3451ddc14203 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:26.891267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47238e92-6230-4793-8791-3451ddc14203 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:26.891547 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47238e92-6230-4793-8791-3451ddc14203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:26.894963 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0415ae6b1f34ff1bbe4b75725d66ced', 'request_id': 'req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.902077 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0415ae6b1f34ff1bbe4b75725d66ced', 'request_id': 'req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.909900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0415ae6b1f34ff1bbe4b75725d66ced', 'request_id': 'req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.910147 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-47238e92-6230-4793-8791-3451ddc14203 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:26.911341 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-47238e92-6230-4793-8791-3451ddc14203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:26.922946 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] The token cache is being invalidate because project d0415ae6b1f34ff1bbe4b75725d66ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:26.923237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.923987 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-908b4ccf-ceee-4dd2-ae69-5c8ba8a2b6f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:26.925614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1366/2761] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:26 2022] DELETE /identity/v3/projects/d0415ae6b1f34ff1bbe4b75725d66ced => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:26.931578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe432656-45ad-47cc-8cfc-95c241f163ed None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:26.931865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe432656-45ad-47cc-8cfc-95c241f163ed None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:26.932255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe432656-45ad-47cc-8cfc-95c241f163ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:26.950644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-fe432656-45ad-47cc-8cfc-95c241f163ed None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:26.951833 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-fe432656-45ad-47cc-8cfc-95c241f163ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:26.957700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:26.979181 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1396/2762] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:26.988713 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47238e92-6230-4793-8791-3451ddc14203 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:26.990050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-47238e92-6230-4793-8791-3451ddc14203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:27.019549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1367/2763] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:27.028493 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b945755-0cc3-4b0a-8898-d0e13394d196 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.028816 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b945755-0cc3-4b0a-8898-d0e13394d196 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.029145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b945755-0cc3-4b0a-8898-d0e13394d196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.048749 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-5b945755-0cc3-4b0a-8898-d0e13394d196 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:27.049933 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-5b945755-0cc3-4b0a-8898-d0e13394d196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:27.053239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47238e92-6230-4793-8791-3451ddc14203 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.057507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97003ddb-ce21-439e-8185-d39629212f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.058025 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1368/2764] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:27.058148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97003ddb-ce21-439e-8185-d39629212f7a service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.058429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97003ddb-ce21-439e-8185-d39629212f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.058742 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97003ddb-ce21-439e-8185-d39629212f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.062078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b945755-0cc3-4b0a-8898-d0e13394d196 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:27.064731 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:27.069664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97003ddb-ce21-439e-8185-d39629212f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.071264 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97003ddb-ce21-439e-8185-d39629212f7a service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.072716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b945755-0cc3-4b0a-8898-d0e13394d196 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.077221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa03ba8-956a-4537-85b7-f79a298d0a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.077908 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa03ba8-956a-4537-85b7-f79a298d0a13 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.078194 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa03ba8-956a-4537-85b7-f79a298d0a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.078553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa03ba8-956a-4537-85b7-f79a298d0a13 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.079282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa03ba8-956a-4537-85b7-f79a298d0a13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:27.079595 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa03ba8-956a-4537-85b7-f79a298d0a13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.081209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa03ba8-956a-4537-85b7-f79a298d0a13 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.083674 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1397/2765] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:27.100337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1369/2766] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:56:27.101913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97003ddb-ce21-439e-8185-d39629212f7a service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:27.112354 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97003ddb-ce21-439e-8185-d39629212f7a service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.117335 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1ab4e-78db-42e4-92b6-fe04cdf7c750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.117984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1ab4e-78db-42e4-92b6-fe04cdf7c750 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.118268 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1ab4e-78db-42e4-92b6-fe04cdf7c750 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.118702 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1ab4e-78db-42e4-92b6-fe04cdf7c750 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.119828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1ab4e-78db-42e4-92b6-fe04cdf7c750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.121772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1ab4e-78db-42e4-92b6-fe04cdf7c750 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.149405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1398/2767] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:27 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:56:27.155356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa03ba8-956a-4537-85b7-f79a298d0a13 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:27.169313 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa03ba8-956a-4537-85b7-f79a298d0a13 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.228697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adebe3-6242-4a6d-bf32-605a835caba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.229635 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adebe3-6242-4a6d-bf32-605a835caba0 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.229918 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adebe3-6242-4a6d-bf32-605a835caba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.230192 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adebe3-6242-4a6d-bf32-605a835caba0 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.230880 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adebe3-6242-4a6d-bf32-605a835caba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.232802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6adebe3-6242-4a6d-bf32-605a835caba0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.263549 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1370/2768] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:27 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:27.264743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1ab4e-78db-42e4-92b6-fe04cdf7c750 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:27.275907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1ab4e-78db-42e4-92b6-fe04cdf7c750 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.280535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ca0e8-2363-43dd-8349-f4fbe71f78e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.281227 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca0e8-2363-43dd-8349-f4fbe71f78e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.281510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca0e8-2363-43dd-8349-f4fbe71f78e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.281786 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca0e8-2363-43dd-8349-f4fbe71f78e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.282507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ca0e8-2363-43dd-8349-f4fbe71f78e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.284106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ca0e8-2363-43dd-8349-f4fbe71f78e3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.295296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1399/2769] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:27.299244 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adebe3-6242-4a6d-bf32-605a835caba0 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:27.309964 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adebe3-6242-4a6d-bf32-605a835caba0 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.315004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191d3e7-8824-4cbc-992c-e17afc6bf75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.315700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191d3e7-8824-4cbc-992c-e17afc6bf75e admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.315981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191d3e7-8824-4cbc-992c-e17afc6bf75e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.316255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191d3e7-8824-4cbc-992c-e17afc6bf75e admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.317014 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191d3e7-8824-4cbc-992c-e17afc6bf75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.318601 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191d3e7-8824-4cbc-992c-e17afc6bf75e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.329694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1371/2770] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:27.333690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ca0e8-2363-43dd-8349-f4fbe71f78e3 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:27.344229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ca0e8-2363-43dd-8349-f4fbe71f78e3 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.348975 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ace610-3a3d-416e-aaf0-6b121a5f138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.349861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ace610-3a3d-416e-aaf0-6b121a5f138a admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.350165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ace610-3a3d-416e-aaf0-6b121a5f138a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.350423 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ace610-3a3d-416e-aaf0-6b121a5f138a admin admin] PATH_INFO: `/v3/projects/ad17bbb802f040508c4f2457ea2199dd/users/0ed1b317f5674c6da179e04a7b4a08e4/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.366789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ace610-3a3d-416e-aaf0-6b121a5f138a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad17bbb802f040508c4f2457ea2199dd, user_id=0ed1b317f5674c6da179e04a7b4a08e4, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.369832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ace610-3a3d-416e-aaf0-6b121a5f138a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.370674 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191d3e7-8824-4cbc-992c-e17afc6bf75e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:27.373402 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f191d3e7-8824-4cbc-992c-e17afc6bf75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:27.388119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191d3e7-8824-4cbc-992c-e17afc6bf75e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.400743 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1400/2771] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:27 2022] PUT /identity/v3/projects/ad17bbb802f040508c4f2457ea2199dd/users/0ed1b317f5674c6da179e04a7b4a08e4/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:27.411604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559100-88c0-4445-abc2-ec91c5b02b7e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.411954 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559100-88c0-4445-abc2-ec91c5b02b7e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.412306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75559100-88c0-4445-abc2-ec91c5b02b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.442502 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-75559100-88c0-4445-abc2-ec91c5b02b7e None None] MFA Rules not processed for user `0ed1b317f5674c6da179e04a7b4a08e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:27.443750 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-75559100-88c0-4445-abc2-ec91c5b02b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:27.463854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.464777 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.465040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.465322 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] PATH_INFO: `/v3/projects/40c3d04f5ceb4b57ae8e560988388c60` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.466588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40c3d04f5ceb4b57ae8e560988388c60)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.468542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.481944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:27.518947 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1401/2772] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:27.520712 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40c3d04f5ceb4b57ae8e560988388c60', 'request_id': 'req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.527686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40c3d04f5ceb4b57ae8e560988388c60', 'request_id': 'req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.531616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559100-88c0-4445-abc2-ec91c5b02b7e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:27.534026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-75559100-88c0-4445-abc2-ec91c5b02b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:27.535237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40c3d04f5ceb4b57ae8e560988388c60', 'request_id': 'req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.545961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] The token cache is being invalidate because project 40c3d04f5ceb4b57ae8e560988388c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:27.546249 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.546886 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.548481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1372/2773] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:27 2022] DELETE /identity/v3/projects/40c3d04f5ceb4b57ae8e560988388c60 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:27.552228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:27.553587 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:27.593531 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75559100-88c0-4445-abc2-ec91c5b02b7e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.617344 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a583c-82d4-4767-86ca-4a83aedc0ac9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:27.622151 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.623015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.623297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.623572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] PATH_INFO: `/v3/users/8856cb6a38ed41bca9662dfbcc2c727b` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.624783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8856cb6a38ed41bca9662dfbcc2c727b)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.626686 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.636189 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:27.648034 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71978a7-1b01-48f8-b5aa-2701515bd0a0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71e5ef9dc32f494791ba4addb5151232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:27.648801 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71978a7-1b01-48f8-b5aa-2701515bd0a0 service glance] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:27.649083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71978a7-1b01-48f8-b5aa-2701515bd0a0 service glance] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:27.649353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71978a7-1b01-48f8-b5aa-2701515bd0a0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:27.651895 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d71978a7-1b01-48f8-b5aa-2701515bd0a0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:27.675327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8856cb6a38ed41bca9662dfbcc2c727b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.687815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8856cb6a38ed41bca9662dfbcc2c727b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.695390 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8856cb6a38ed41bca9662dfbcc2c727b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.696092 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8856cb6a38ed41bca9662dfbcc2c727b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.703575 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8856cb6a38ed41bca9662dfbcc2c727b'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:27.709612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1373/2774] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:27 2022] DELETE /identity/v3/users/8856cb6a38ed41bca9662dfbcc2c727b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:27.716927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71978a7-1b01-48f8-b5aa-2701515bd0a0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:27.718402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71978a7-1b01-48f8-b5aa-2701515bd0a0 service glance] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:27.727538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1402/2775] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:28.001799 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.003434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:28.034078 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:28.080352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69279b7d-0ed6-46d1-8614-8c9ebeed43f2 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.085303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.086184 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.086459 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.086735 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] PATH_INFO: `/v3/projects/83077bece7854caea7b4791f9b52b525` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.088230 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83077bece7854caea7b4791f9b52b525)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.090157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.139741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83077bece7854caea7b4791f9b52b525', 'request_id': 'req-905b8f5c-983c-47b3-91a8-a7981e4de495'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:28.147029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83077bece7854caea7b4791f9b52b525', 'request_id': 'req-905b8f5c-983c-47b3-91a8-a7981e4de495'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:28.157243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83077bece7854caea7b4791f9b52b525', 'request_id': 'req-905b8f5c-983c-47b3-91a8-a7981e4de495'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:28.170751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] The token cache is being invalidate because project 83077bece7854caea7b4791f9b52b525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:28.171032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:28.171721 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-905b8f5c-983c-47b3-91a8-a7981e4de495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:28.177040 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1374/2776] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:27 2022] DELETE /identity/v3/projects/83077bece7854caea7b4791f9b52b525 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:56:28.178446 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331cb78-eaea-4830-ac42-f9cff674946e None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.178871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331cb78-eaea-4830-ac42-f9cff674946e None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.179013 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331cb78-eaea-4830-ac42-f9cff674946e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.200634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-9331cb78-eaea-4830-ac42-f9cff674946e None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:28.201932 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9331cb78-eaea-4830-ac42-f9cff674946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:28.267445 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1403/2777] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:28.276958 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811310f6-5e81-4eac-bb01-f26f1f96a454 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.277246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811310f6-5e81-4eac-bb01-f26f1f96a454 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.277563 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811310f6-5e81-4eac-bb01-f26f1f96a454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.296783 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-811310f6-5e81-4eac-bb01-f26f1f96a454 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:28.297975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-811310f6-5e81-4eac-bb01-f26f1f96a454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:28.307117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1375/2778] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:28.311322 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331cb78-eaea-4830-ac42-f9cff674946e None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.321162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331cb78-eaea-4830-ac42-f9cff674946e None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.326500 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef212be-82b8-4d89-8321-3cab2ae3e92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.327148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef212be-82b8-4d89-8321-3cab2ae3e92d admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.327429 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef212be-82b8-4d89-8321-3cab2ae3e92d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.327704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef212be-82b8-4d89-8321-3cab2ae3e92d admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.328422 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef212be-82b8-4d89-8321-3cab2ae3e92d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:28.328774 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef212be-82b8-4d89-8321-3cab2ae3e92d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.330435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef212be-82b8-4d89-8321-3cab2ae3e92d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.342213 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1404/2779] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:28.346221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811310f6-5e81-4eac-bb01-f26f1f96a454 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.355438 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811310f6-5e81-4eac-bb01-f26f1f96a454 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.360145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4bbae-8e65-4504-b5c0-eadd90190dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.360828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4bbae-8e65-4504-b5c0-eadd90190dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.361110 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4bbae-8e65-4504-b5c0-eadd90190dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.361387 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4bbae-8e65-4504-b5c0-eadd90190dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.362097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4bbae-8e65-4504-b5c0-eadd90190dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.363982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4bbae-8e65-4504-b5c0-eadd90190dc0 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.391098 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1376/2780] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:28 2022] POST /identity/v3/projects => generated 409 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:28.395610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef212be-82b8-4d89-8321-3cab2ae3e92d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.406177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef212be-82b8-4d89-8321-3cab2ae3e92d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.464793 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d18db-a118-48c3-811f-012e03f2fd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.465370 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d18db-a118-48c3-811f-012e03f2fd3e admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.465776 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d18db-a118-48c3-811f-012e03f2fd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.466062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d18db-a118-48c3-811f-012e03f2fd3e admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.466752 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d18db-a118-48c3-811f-012e03f2fd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.468818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d18db-a118-48c3-811f-012e03f2fd3e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.492166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1405/2781] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:28 2022] POST /identity/v3/users => generated 305 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:28.494468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4bbae-8e65-4504-b5c0-eadd90190dc0 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.504887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4bbae-8e65-4504-b5c0-eadd90190dc0 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.509998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a1d0e-3f45-4aeb-9ec1-9cb60d5c248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.510657 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1d0e-3f45-4aeb-9ec1-9cb60d5c248a admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.510941 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1d0e-3f45-4aeb-9ec1-9cb60d5c248a admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.511221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1d0e-3f45-4aeb-9ec1-9cb60d5c248a admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.511957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a1d0e-3f45-4aeb-9ec1-9cb60d5c248a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.513583 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a1d0e-3f45-4aeb-9ec1-9cb60d5c248a admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.524355 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1377/2782] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:28.528946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d18db-a118-48c3-811f-012e03f2fd3e admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.538788 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d18db-a118-48c3-811f-012e03f2fd3e admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.543934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e27147-16f4-43c8-ad74-117ae89555b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.545177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e27147-16f4-43c8-ad74-117ae89555b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.545177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e27147-16f4-43c8-ad74-117ae89555b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.545177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e27147-16f4-43c8-ad74-117ae89555b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.545968 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e27147-16f4-43c8-ad74-117ae89555b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.547717 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e27147-16f4-43c8-ad74-117ae89555b1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.559410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1406/2783] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:28.563898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a1d0e-3f45-4aeb-9ec1-9cb60d5c248a admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.575089 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a1d0e-3f45-4aeb-9ec1-9cb60d5c248a admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.579982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bd5fd-7cc1-459f-9da8-976f3c59270b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.580909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bd5fd-7cc1-459f-9da8-976f3c59270b admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.581190 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bd5fd-7cc1-459f-9da8-976f3c59270b admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.581469 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bd5fd-7cc1-459f-9da8-976f3c59270b admin admin] PATH_INFO: `/v3/projects/108a6f8edfef4694b9e7eb7f2e852d04/users/154b1b97ac194527bc508dbb745a0bb0/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.597677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bd5fd-7cc1-459f-9da8-976f3c59270b admin admin] RBAC: Authorizing `identity:create_grant(project_id=108a6f8edfef4694b9e7eb7f2e852d04, user_id=154b1b97ac194527bc508dbb745a0bb0, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.600327 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bd5fd-7cc1-459f-9da8-976f3c59270b admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.614592 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1378/2784] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:28 2022] PUT /identity/v3/projects/108a6f8edfef4694b9e7eb7f2e852d04/users/154b1b97ac194527bc508dbb745a0bb0/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:28.625044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da4c60-0dba-42e6-b91d-85072dffd406 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.625276 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da4c60-0dba-42e6-b91d-85072dffd406 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.625646 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da4c60-0dba-42e6-b91d-85072dffd406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.632097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:28.656972 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-32da4c60-0dba-42e6-b91d-85072dffd406 None None] MFA Rules not processed for user `154b1b97ac194527bc508dbb745a0bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:28.658353 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-32da4c60-0dba-42e6-b91d-85072dffd406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:28.726631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1407/2785] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:28 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:28.731127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bd5fd-7cc1-459f-9da8-976f3c59270b admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.783130 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:28.800006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bd5fd-7cc1-459f-9da8-976f3c59270b admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.804789 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7b9cc-2899-4181-83b1-9714a53bd0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.805479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7b9cc-2899-4181-83b1-9714a53bd0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.805865 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7b9cc-2899-4181-83b1-9714a53bd0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.806189 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7b9cc-2899-4181-83b1-9714a53bd0e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.806892 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7b9cc-2899-4181-83b1-9714a53bd0e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.808814 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7b9cc-2899-4181-83b1-9714a53bd0e7 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.834824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1379/2786] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:28 2022] POST /identity/v3/projects => generated 407 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:28.839714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da4c60-0dba-42e6-b91d-85072dffd406 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.850337 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da4c60-0dba-42e6-b91d-85072dffd406 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.913861 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec01e20-1e38-4dfb-855f-8eadc9b707f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.914448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec01e20-1e38-4dfb-855f-8eadc9b707f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.914729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec01e20-1e38-4dfb-855f-8eadc9b707f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.915003 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec01e20-1e38-4dfb-855f-8eadc9b707f1 admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.915852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec01e20-1e38-4dfb-855f-8eadc9b707f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.917828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec01e20-1e38-4dfb-855f-8eadc9b707f1 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.942576 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1408/2787] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:28 2022] POST /identity/v3/users => generated 310 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:28.947161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7b9cc-2899-4181-83b1-9714a53bd0e7 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.957995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7b9cc-2899-4181-83b1-9714a53bd0e7 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.963279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c7163-c40b-4646-ac47-f9cb78c343d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.963539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c7163-c40b-4646-ac47-f9cb78c343d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.963828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c7163-c40b-4646-ac47-f9cb78c343d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.964111 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c7163-c40b-4646-ac47-f9cb78c343d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:28.964885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c7163-c40b-4646-ac47-f9cb78c343d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:28.966477 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c7163-c40b-4646-ac47-f9cb78c343d6 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:28.977785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1380/2788] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:28.982040 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec01e20-1e38-4dfb-855f-8eadc9b707f1 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:28.992073 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec01e20-1e38-4dfb-855f-8eadc9b707f1 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:28.997265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693706d-9ae9-42f1-b037-f01b3cadc3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:28.998131 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693706d-9ae9-42f1-b037-f01b3cadc3da admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:28.998411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693706d-9ae9-42f1-b037-f01b3cadc3da admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:28.998687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693706d-9ae9-42f1-b037-f01b3cadc3da admin admin] PATH_INFO: `/v3/projects/99239414e5184cf282f8f28f6101fef5/users/f715b83b1f2549f6a4b0f3a9547370b1/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:29.015790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693706d-9ae9-42f1-b037-f01b3cadc3da admin admin] RBAC: Authorizing `identity:create_grant(project_id=99239414e5184cf282f8f28f6101fef5, user_id=f715b83b1f2549f6a4b0f3a9547370b1, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:29.021630 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693706d-9ae9-42f1-b037-f01b3cadc3da admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:29.035193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1409/2789] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:28 2022] PUT /identity/v3/projects/99239414e5184cf282f8f28f6101fef5/users/f715b83b1f2549f6a4b0f3a9547370b1/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:29.039354 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c7163-c40b-4646-ac47-f9cb78c343d6 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:29.050030 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c7163-c40b-4646-ac47-f9cb78c343d6 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:29.107676 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f2831-48fb-40c1-acf6-eb4b7bcf50be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:29.108351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f2831-48fb-40c1-acf6-eb4b7bcf50be admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:29.108672 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f2831-48fb-40c1-acf6-eb4b7bcf50be admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:29.108952 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f2831-48fb-40c1-acf6-eb4b7bcf50be admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:29.109688 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f2831-48fb-40c1-acf6-eb4b7bcf50be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:29.111284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f2831-48fb-40c1-acf6-eb4b7bcf50be admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:29.122188 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1381/2790] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:29.126557 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693706d-9ae9-42f1-b037-f01b3cadc3da admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:29.137604 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693706d-9ae9-42f1-b037-f01b3cadc3da admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:29.142104 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93356ed6-7e38-416c-bb94-5ca8268d128b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:29.143233 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93356ed6-7e38-416c-bb94-5ca8268d128b admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:29.143516 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93356ed6-7e38-416c-bb94-5ca8268d128b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:29.143790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93356ed6-7e38-416c-bb94-5ca8268d128b admin admin] PATH_INFO: `/v3/projects/99239414e5184cf282f8f28f6101fef5/users/f715b83b1f2549f6a4b0f3a9547370b1/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:29.145759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93356ed6-7e38-416c-bb94-5ca8268d128b admin admin] RBAC: Authorizing `identity:create_grant(project_id=99239414e5184cf282f8f28f6101fef5, user_id=f715b83b1f2549f6a4b0f3a9547370b1, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:29.148425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93356ed6-7e38-416c-bb94-5ca8268d128b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:29.159453 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1410/2791] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:29 2022] PUT /identity/v3/projects/99239414e5184cf282f8f28f6101fef5/users/f715b83b1f2549f6a4b0f3a9547370b1/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:29.171586 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1738f0-60ca-4405-9c55-3bc4323309f4 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:29.171808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1738f0-60ca-4405-9c55-3bc4323309f4 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:29.172054 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1738f0-60ca-4405-9c55-3bc4323309f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:29.200921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-df1738f0-60ca-4405-9c55-3bc4323309f4 None None] MFA Rules not processed for user `f715b83b1f2549f6a4b0f3a9547370b1`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:29.202128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-df1738f0-60ca-4405-9c55-3bc4323309f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:29.276962 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1382/2792] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:29 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:29.287450 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93356ed6-7e38-416c-bb94-5ca8268d128b admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:29.289166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-93356ed6-7e38-416c-bb94-5ca8268d128b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:29.337613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:29.350352 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93356ed6-7e38-416c-bb94-5ca8268d128b admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:29.354650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee1749-5aae-4d4b-8b79-f127fedb829b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:29.355291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee1749-5aae-4d4b-8b79-f127fedb829b service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:29.355570 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee1749-5aae-4d4b-8b79-f127fedb829b service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:29.355843 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee1749-5aae-4d4b-8b79-f127fedb829b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:29.365044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee1749-5aae-4d4b-8b79-f127fedb829b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:29.366472 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee1749-5aae-4d4b-8b79-f127fedb829b service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:29.376818 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1411/2793] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:29 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:29.530961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1738f0-60ca-4405-9c55-3bc4323309f4 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:29.541106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1738f0-60ca-4405-9c55-3bc4323309f4 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:29.545034 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388047c-104f-4744-84d1-7c695b332fee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:29.545736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388047c-104f-4744-84d1-7c695b332fee service cinder] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:29.546023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388047c-104f-4744-84d1-7c695b332fee service cinder] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:29.546308 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388047c-104f-4744-84d1-7c695b332fee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:29.590566 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:29.608502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388047c-104f-4744-84d1-7c695b332fee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:29.610004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388047c-104f-4744-84d1-7c695b332fee service cinder] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:29.620806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1383/2794] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:36.930652 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee1749-5aae-4d4b-8b79-f127fedb829b service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:36.995170 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee1749-5aae-4d4b-8b79-f127fedb829b service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:36.999741 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:37.000637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:37.000926 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:37.001203 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] PATH_INFO: `/v3/users/154b1b97ac194527bc508dbb745a0bb0` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:37.002387 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] RBAC: Authorizing `identity:delete_user(user_id=154b1b97ac194527bc508dbb745a0bb0)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:37.004296 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:37.051396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154b1b97ac194527bc508dbb745a0bb0'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.064106 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154b1b97ac194527bc508dbb745a0bb0'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.064846 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154b1b97ac194527bc508dbb745a0bb0'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.065456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154b1b97ac194527bc508dbb745a0bb0'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.072884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154b1b97ac194527bc508dbb745a0bb0'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.086128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1412/2795] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:36 2022] DELETE /identity/v3/users/154b1b97ac194527bc508dbb745a0bb0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:37.092063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27f856-ef3a-499f-a5b7-3de38d7dcb4c None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:37.092539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27f856-ef3a-499f-a5b7-3de38d7dcb4c None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:37.092785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27f856-ef3a-499f-a5b7-3de38d7dcb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:37.111111 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-8b27f856-ef3a-499f-a5b7-3de38d7dcb4c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:37.112324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8b27f856-ef3a-499f-a5b7-3de38d7dcb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:37.175233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1384/2796] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:37.185501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:37.186973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:37.243664 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432e04a-0f7b-4aaf-89c2-42ca1106444c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:37.248043 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf16077-c9ec-4869-84da-c7ee909e4b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:37.248810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf16077-c9ec-4869-84da-c7ee909e4b17 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:37.249087 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf16077-c9ec-4869-84da-c7ee909e4b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:37.249371 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf16077-c9ec-4869-84da-c7ee909e4b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:37.259481 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf16077-c9ec-4869-84da-c7ee909e4b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:37.260982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf16077-c9ec-4869-84da-c7ee909e4b17 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:37.271790 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1413/2797] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:37.556667 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27f856-ef3a-499f-a5b7-3de38d7dcb4c None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:37.558123 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8b27f856-ef3a-499f-a5b7-3de38d7dcb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:37.573369 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27f856-ef3a-499f-a5b7-3de38d7dcb4c None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:37.578917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:37.579813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:37.580097 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:37.580409 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] PATH_INFO: `/v3/projects/108a6f8edfef4694b9e7eb7f2e852d04` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:37.581607 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108a6f8edfef4694b9e7eb7f2e852d04)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:37.583679 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:37.636311 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108a6f8edfef4694b9e7eb7f2e852d04', 'request_id': 'req-7648d951-c722-4367-8689-5b516b1f7799'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.643896 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108a6f8edfef4694b9e7eb7f2e852d04', 'request_id': 'req-7648d951-c722-4367-8689-5b516b1f7799'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.651767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108a6f8edfef4694b9e7eb7f2e852d04', 'request_id': 'req-7648d951-c722-4367-8689-5b516b1f7799'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.663581 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] The token cache is being invalidate because project 108a6f8edfef4694b9e7eb7f2e852d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:37.664508 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.665255 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.667039 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1385/2798] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:37 2022] DELETE /identity/v3/projects/108a6f8edfef4694b9e7eb7f2e852d04 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:37.671039 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf16077-c9ec-4869-84da-c7ee909e4b17 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:37.680969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf16077-c9ec-4869-84da-c7ee909e4b17 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:37.685745 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:37.744259 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:37.745141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:37.745619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:37.745908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] PATH_INFO: `/v3/users/f715b83b1f2549f6a4b0f3a9547370b1` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:37.747204 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f715b83b1f2549f6a4b0f3a9547370b1)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:37.750077 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:37.798225 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f715b83b1f2549f6a4b0f3a9547370b1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.811161 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f715b83b1f2549f6a4b0f3a9547370b1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.811878 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f715b83b1f2549f6a4b0f3a9547370b1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.812474 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f715b83b1f2549f6a4b0f3a9547370b1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.820342 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-eb7d6183-43c7-4237-b7c1-65c0b2f27ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f715b83b1f2549f6a4b0f3a9547370b1'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:37.838061 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1414/2799] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:37 2022] DELETE /identity/v3/users/f715b83b1f2549f6a4b0f3a9547370b1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:38.118745 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:38.120576 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:38.136788 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d951-c722-4367-8689-5b516b1f7799 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:38.202044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:38.202819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.203099 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.203374 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] PATH_INFO: `/v3/projects/99239414e5184cf282f8f28f6101fef5` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.204538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=99239414e5184cf282f8f28f6101fef5)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:38.206641 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:38.213760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:38.254856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99239414e5184cf282f8f28f6101fef5', 'request_id': 'req-804c11b2-38fa-4686-8170-81237c810fdd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:38.261658 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99239414e5184cf282f8f28f6101fef5', 'request_id': 'req-804c11b2-38fa-4686-8170-81237c810fdd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:38.270342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99239414e5184cf282f8f28f6101fef5', 'request_id': 'req-804c11b2-38fa-4686-8170-81237c810fdd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:38.282004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] The token cache is being invalidate because project 99239414e5184cf282f8f28f6101fef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:38.282295 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:38.282943 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-804c11b2-38fa-4686-8170-81237c810fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:38.284510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1386/2800] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:38 2022] DELETE /identity/v3/projects/99239414e5184cf282f8f28f6101fef5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:38.289605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fe1da-b013-41b8-aab6-1206fb3d831c None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.290201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fe1da-b013-41b8-aab6-1206fb3d831c None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.290201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fe1da-b013-41b8-aab6-1206fb3d831c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.308329 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3e9fe1da-b013-41b8-aab6-1206fb3d831c None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:38.309580 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3e9fe1da-b013-41b8-aab6-1206fb3d831c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:38.370996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1415/2801] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:38.380118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa4d6f-1a26-4515-a1fe-41a42784dcab None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.380407 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa4d6f-1a26-4515-a1fe-41a42784dcab None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.380751 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa4d6f-1a26-4515-a1fe-41a42784dcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.399779 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f6fa4d6f-1a26-4515-a1fe-41a42784dcab None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:38.400999 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f6fa4d6f-1a26-4515-a1fe-41a42784dcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:38.410432 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1387/2802] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:38.414715 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fe1da-b013-41b8-aab6-1206fb3d831c None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:38.424862 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fe1da-b013-41b8-aab6-1206fb3d831c None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:38.429425 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868e412-0952-418f-a161-3905c026fc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:38.430109 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868e412-0952-418f-a161-3905c026fc3b admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.430395 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868e412-0952-418f-a161-3905c026fc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.430673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868e412-0952-418f-a161-3905c026fc3b admin admin] PATH_INFO: `/v3/domains` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.431431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868e412-0952-418f-a161-3905c026fc3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:56:38.431739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868e412-0952-418f-a161-3905c026fc3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:38.433393 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868e412-0952-418f-a161-3905c026fc3b admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:38.445094 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1416/2803] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:56:38.449546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa4d6f-1a26-4515-a1fe-41a42784dcab None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:38.460027 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa4d6f-1a26-4515-a1fe-41a42784dcab None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:38.464866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e6d7a-6cb9-44d8-b790-a5e50c96d570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:38.465521 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e6d7a-6cb9-44d8-b790-a5e50c96d570 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.465805 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e6d7a-6cb9-44d8-b790-a5e50c96d570 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.466204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e6d7a-6cb9-44d8-b790-a5e50c96d570 admin admin] PATH_INFO: `/v3/projects` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.467399 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e6d7a-6cb9-44d8-b790-a5e50c96d570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:38.469336 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e6d7a-6cb9-44d8-b790-a5e50c96d570 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:38.494431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1388/2804] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:56:38 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:38.498590 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868e412-0952-418f-a161-3905c026fc3b admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:38.508485 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868e412-0952-418f-a161-3905c026fc3b admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:38.521594 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:38.569334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797f3df-8cd5-4bde-9cfb-1f7034be429d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:38.570021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797f3df-8cd5-4bde-9cfb-1f7034be429d admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.570301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797f3df-8cd5-4bde-9cfb-1f7034be429d admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.570577 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1797f3df-8cd5-4bde-9cfb-1f7034be429d admin admin] PATH_INFO: `/v3/users` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.571383 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797f3df-8cd5-4bde-9cfb-1f7034be429d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:38.573331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1797f3df-8cd5-4bde-9cfb-1f7034be429d admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:38.598117 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1417/2805] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:56:38 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:56:38.599478 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e6d7a-6cb9-44d8-b790-a5e50c96d570 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:38.610683 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e6d7a-6cb9-44d8-b790-a5e50c96d570 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:38.615248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797f3df-8cd5-4bde-9cfb-1f7034be429d admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:38.615450 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192da514-4d8c-455a-bbcb-7b46d390cd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:38.616115 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192da514-4d8c-455a-bbcb-7b46d390cd55 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.616456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192da514-4d8c-455a-bbcb-7b46d390cd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.616700 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192da514-4d8c-455a-bbcb-7b46d390cd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.617701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1797f3df-8cd5-4bde-9cfb-1f7034be429d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:38.617831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192da514-4d8c-455a-bbcb-7b46d390cd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:38.619096 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192da514-4d8c-455a-bbcb-7b46d390cd55 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:38.633178 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1389/2806] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:56:38.634266 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192da514-4d8c-455a-bbcb-7b46d390cd55 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:38.646363 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192da514-4d8c-455a-bbcb-7b46d390cd55 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:38.651246 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389829f-8437-42e1-9aab-61762b0d4320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:38.651947 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389829f-8437-42e1-9aab-61762b0d4320 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.652233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389829f-8437-42e1-9aab-61762b0d4320 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.652498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389829f-8437-42e1-9aab-61762b0d4320 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.653329 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389829f-8437-42e1-9aab-61762b0d4320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:38.655640 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389829f-8437-42e1-9aab-61762b0d4320 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:38.668553 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1390/2807] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:56:38.670494 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389829f-8437-42e1-9aab-61762b0d4320 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:38.682957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389829f-8437-42e1-9aab-61762b0d4320 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:38.688875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306348d-6044-4570-8661-85f80b8e4419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:38.690127 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306348d-6044-4570-8661-85f80b8e4419 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.690678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306348d-6044-4570-8661-85f80b8e4419 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.690961 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306348d-6044-4570-8661-85f80b8e4419 admin admin] PATH_INFO: `/v3/projects/841ece12881d4b998e5c8e7044cb0b5b/users/f0a2f0122b654850b26bddc1838903aa/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.698396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1797f3df-8cd5-4bde-9cfb-1f7034be429d admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:38.706816 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:38.707548 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.707906 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.708257 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.710199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:38.711565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306348d-6044-4570-8661-85f80b8e4419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841ece12881d4b998e5c8e7044cb0b5b, user_id=f0a2f0122b654850b26bddc1838903aa, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:38.714265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306348d-6044-4570-8661-85f80b8e4419 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:38.727063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:38.728646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1391/2808] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:56:38 2022] PUT /identity/v3/projects/841ece12881d4b998e5c8e7044cb0b5b/users/f0a2f0122b654850b26bddc1838903aa/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:38.728820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:38.738914 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8891845-70ed-4738-bc77-36dad8b774fd None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.739214 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8891845-70ed-4738-bc77-36dad8b774fd None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.739497 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8891845-70ed-4738-bc77-36dad8b774fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.739928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1418/2809] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 23:56:38.769885 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-a8891845-70ed-4738-bc77-36dad8b774fd None None] MFA Rules not processed for user `f0a2f0122b654850b26bddc1838903aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:38.771095 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a8891845-70ed-4738-bc77-36dad8b774fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:38.837832 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1392/2810] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 23:56:38.849682 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:38.851086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:38.908315 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c578a-0b2a-43b1-bd42-f63345ed72e3 service placement] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:38.964748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b265e64-ae6d-47c6-b034-bb3bff464603 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71e5ef9dc32f494791ba4addb5151232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:38.965405 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b265e64-ae6d-47c6-b034-bb3bff464603 service glance] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:38.965687 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b265e64-ae6d-47c6-b034-bb3bff464603 service glance] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:38.965971 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b265e64-ae6d-47c6-b034-bb3bff464603 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:38.971989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:38.976120 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b265e64-ae6d-47c6-b034-bb3bff464603 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:38.977778 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b265e64-ae6d-47c6-b034-bb3bff464603 service glance] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:38.989045 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1419/2811] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:48.834762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:48.984650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:56:55.477162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8891845-70ed-4738-bc77-36dad8b774fd None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:55.479031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-a8891845-70ed-4738-bc77-36dad8b774fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:55.548980 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8891845-70ed-4738-bc77-36dad8b774fd None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:55.553536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:55.554408 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:55.554689 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:55.554966 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] PATH_INFO: `/v3/users/0ed1b317f5674c6da179e04a7b4a08e4` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:55.556715 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed1b317f5674c6da179e04a7b4a08e4)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:55.558714 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:55.607279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed1b317f5674c6da179e04a7b4a08e4'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:55.620291 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed1b317f5674c6da179e04a7b4a08e4'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:55.627651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed1b317f5674c6da179e04a7b4a08e4'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:55.628290 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed1b317f5674c6da179e04a7b4a08e4'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:55.635445 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed1b317f5674c6da179e04a7b4a08e4'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:55.642501 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1393/2812] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:56:55 2022] DELETE /identity/v3/users/0ed1b317f5674c6da179e04a7b4a08e4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:56:55.646990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd75f1-5cf9-4037-a29f-a35e73086d45 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:55.647493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd75f1-5cf9-4037-a29f-a35e73086d45 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:55.647603 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd75f1-5cf9-4037-a29f-a35e73086d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:55.666112 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-48dd75f1-5cf9-4037-a29f-a35e73086d45 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:56:55.667297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-48dd75f1-5cf9-4037-a29f-a35e73086d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:55.727739 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1420/2813] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:56:55.737376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:55.738806 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:55.797476 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb24cbd-2c77-4c2f-9f96-43bb8324ec03 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:55.801131 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d67c8-eb43-40bc-ad18-49045e3588db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:55.801780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d67c8-eb43-40bc-ad18-49045e3588db service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:55.802062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d67c8-eb43-40bc-ad18-49045e3588db service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:55.802342 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d67c8-eb43-40bc-ad18-49045e3588db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:55.811338 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d67c8-eb43-40bc-ad18-49045e3588db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:55.813305 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d67c8-eb43-40bc-ad18-49045e3588db service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:55.823119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1394/2814] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:56:56.092113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd75f1-5cf9-4037-a29f-a35e73086d45 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:56:56.093711 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-48dd75f1-5cf9-4037-a29f-a35e73086d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:56:56.105700 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd75f1-5cf9-4037-a29f-a35e73086d45 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:56:56.113993 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06008204-292d-4907-92c9-154162b133ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:56:56.114871 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06008204-292d-4907-92c9-154162b133ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:56:56.115171 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06008204-292d-4907-92c9-154162b133ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:56:56.115569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06008204-292d-4907-92c9-154162b133ed admin admin] PATH_INFO: `/v3/projects/ad17bbb802f040508c4f2457ea2199dd` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:56:56.116654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06008204-292d-4907-92c9-154162b133ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad17bbb802f040508c4f2457ea2199dd)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:56:56.118595 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06008204-292d-4907-92c9-154162b133ed admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:56:56.169998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-06008204-292d-4907-92c9-154162b133ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad17bbb802f040508c4f2457ea2199dd', 'request_id': 'req-06008204-292d-4907-92c9-154162b133ed'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:56.177966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-06008204-292d-4907-92c9-154162b133ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad17bbb802f040508c4f2457ea2199dd', 'request_id': 'req-06008204-292d-4907-92c9-154162b133ed'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:56.184100 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-06008204-292d-4907-92c9-154162b133ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad17bbb802f040508c4f2457ea2199dd', 'request_id': 'req-06008204-292d-4907-92c9-154162b133ed'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:56.194940 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-06008204-292d-4907-92c9-154162b133ed admin admin] The token cache is being invalidate because project ad17bbb802f040508c4f2457ea2199dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:56:56.195226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-06008204-292d-4907-92c9-154162b133ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:56.195856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-06008204-292d-4907-92c9-154162b133ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:56:56.197431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1421/2815] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:56:56 2022] DELETE /identity/v3/projects/ad17bbb802f040508c4f2457ea2199dd => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:57:05.818681 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:06.192002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:11.085778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d67c8-eb43-40bc-ad18-49045e3588db service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:11.087198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e94d67c8-eb43-40bc-ad18-49045e3588db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:11.099808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d67c8-eb43-40bc-ad18-49045e3588db service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:11.155695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:11.156565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:11.156868 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:11.157138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] PATH_INFO: `/v3/users/f0a2f0122b654850b26bddc1838903aa` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:11.158303 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a2f0122b654850b26bddc1838903aa)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:11.160522 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:11.207646 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a2f0122b654850b26bddc1838903aa'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.219332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a2f0122b654850b26bddc1838903aa'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.227596 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a2f0122b654850b26bddc1838903aa'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.228273 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a2f0122b654850b26bddc1838903aa'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.238224 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a2f0122b654850b26bddc1838903aa'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.245426 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1395/2816] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:57:11 2022] DELETE /identity/v3/users/f0a2f0122b654850b26bddc1838903aa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:57:11.250990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff6606-cbd1-4dee-a6e7-d3c9f7315a00 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:11.251579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff6606-cbd1-4dee-a6e7-d3c9f7315a00 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:11.251689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff6606-cbd1-4dee-a6e7-d3c9f7315a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:11.272534 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3fff6606-cbd1-4dee-a6e7-d3c9f7315a00 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:11.273889 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3fff6606-cbd1-4dee-a6e7-d3c9f7315a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:11.333789 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1422/2817] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:11.344217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:11.345678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:11.404009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5dff7-d042-4be1-95f0-ba3d9b53a9f5 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:11.407680 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d5487-4e53-4059-8518-bd578243bd3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:11.408325 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d5487-4e53-4059-8518-bd578243bd3e service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:11.408638 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d5487-4e53-4059-8518-bd578243bd3e service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:11.409456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d5487-4e53-4059-8518-bd578243bd3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:11.418465 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d5487-4e53-4059-8518-bd578243bd3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:11.419903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d5487-4e53-4059-8518-bd578243bd3e service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:11.430004 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1396/2818] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:57:11.695736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff6606-cbd1-4dee-a6e7-d3c9f7315a00 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:11.697201 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3fff6606-cbd1-4dee-a6e7-d3c9f7315a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:11.712141 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff6606-cbd1-4dee-a6e7-d3c9f7315a00 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:11.717236 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:11.718116 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:11.718396 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:11.718631 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] PATH_INFO: `/v3/projects/841ece12881d4b998e5c8e7044cb0b5b` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:11.719837 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=841ece12881d4b998e5c8e7044cb0b5b)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:11.722241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:11.769105 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841ece12881d4b998e5c8e7044cb0b5b', 'request_id': 'req-e44531f8-51ab-4f62-918e-95ce37527612'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.776135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841ece12881d4b998e5c8e7044cb0b5b', 'request_id': 'req-e44531f8-51ab-4f62-918e-95ce37527612'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.782318 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841ece12881d4b998e5c8e7044cb0b5b', 'request_id': 'req-e44531f8-51ab-4f62-918e-95ce37527612'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.793126 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] The token cache is being invalidate because project 841ece12881d4b998e5c8e7044cb0b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:57:11.793418 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.794055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-e44531f8-51ab-4f62-918e-95ce37527612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:11.795587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1423/2819] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:57:11 2022] DELETE /identity/v3/projects/841ece12881d4b998e5c8e7044cb0b5b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:57:11.800959 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60688ae9-1aa7-452e-b747-62ce67fe0024 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:11.801256 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60688ae9-1aa7-452e-b747-62ce67fe0024 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:11.801542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60688ae9-1aa7-452e-b747-62ce67fe0024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:11.819778 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-60688ae9-1aa7-452e-b747-62ce67fe0024 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:11.821029 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-60688ae9-1aa7-452e-b747-62ce67fe0024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:11.837697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:11.882614 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1397/2820] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:11.891399 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5ae93-ceec-44d3-aafe-174eefab3ecc None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:11.891696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5ae93-ceec-44d3-aafe-174eefab3ecc None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:11.892230 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5ae93-ceec-44d3-aafe-174eefab3ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:11.910454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-e6d5ae93-ceec-44d3-aafe-174eefab3ecc None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:11.911650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e6d5ae93-ceec-44d3-aafe-174eefab3ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:11.919434 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1424/2821] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:11.923456 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60688ae9-1aa7-452e-b747-62ce67fe0024 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:11.933691 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60688ae9-1aa7-452e-b747-62ce67fe0024 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:11.938379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6846c-8ca5-4dce-ab34-ac615eb980c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:11.939036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6846c-8ca5-4dce-ab34-ac615eb980c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:11.939324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6846c-8ca5-4dce-ab34-ac615eb980c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:11.939606 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6846c-8ca5-4dce-ab34-ac615eb980c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:11.940923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6846c-8ca5-4dce-ab34-ac615eb980c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:57:11.941240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6846c-8ca5-4dce-ab34-ac615eb980c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:11.942834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6846c-8ca5-4dce-ab34-ac615eb980c9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:11.954524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1398/2822] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 23:57:11.959062 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5ae93-ceec-44d3-aafe-174eefab3ecc None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:11.968407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5ae93-ceec-44d3-aafe-174eefab3ecc None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:11.973283 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8142bdf-cff8-4b99-87d2-d492a9b000e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:11.973927 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8142bdf-cff8-4b99-87d2-d492a9b000e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:11.974208 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8142bdf-cff8-4b99-87d2-d492a9b000e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:11.974483 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8142bdf-cff8-4b99-87d2-d492a9b000e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:11.975176 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8142bdf-cff8-4b99-87d2-d492a9b000e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:11.977119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8142bdf-cff8-4b99-87d2-d492a9b000e0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:12.002137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1425/2823] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:57:11 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:57:12.006627 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6846c-8ca5-4dce-ab34-ac615eb980c9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:12.016519 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6846c-8ca5-4dce-ab34-ac615eb980c9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:12.072107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bae86a-d56b-4d61-bdb3-466730a89e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:12.072703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bae86a-d56b-4d61-bdb3-466730a89e5e admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:12.072986 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bae86a-d56b-4d61-bdb3-466730a89e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:12.073258 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bae86a-d56b-4d61-bdb3-466730a89e5e admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:12.073945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bae86a-d56b-4d61-bdb3-466730a89e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:12.075834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bae86a-d56b-4d61-bdb3-466730a89e5e admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:12.095261 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1399/2824] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:57:12 2022] POST /identity/v3/users => generated 295 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:57:12.099996 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8142bdf-cff8-4b99-87d2-d492a9b000e0 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:12.110152 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8142bdf-cff8-4b99-87d2-d492a9b000e0 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:12.115135 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc759cd-3d75-457e-86e2-9e92b53039a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:12.115692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc759cd-3d75-457e-86e2-9e92b53039a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:12.115973 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc759cd-3d75-457e-86e2-9e92b53039a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:12.116248 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc759cd-3d75-457e-86e2-9e92b53039a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:12.116998 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc759cd-3d75-457e-86e2-9e92b53039a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:12.118577 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc759cd-3d75-457e-86e2-9e92b53039a5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:12.129263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1426/2825] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:57:12.133137 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bae86a-d56b-4d61-bdb3-466730a89e5e admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:12.144010 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bae86a-d56b-4d61-bdb3-466730a89e5e admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:12.148569 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432428e9-7d57-42df-8391-3c55d927ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:12.149257 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432428e9-7d57-42df-8391-3c55d927ca2d admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:12.149543 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432428e9-7d57-42df-8391-3c55d927ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:12.149853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432428e9-7d57-42df-8391-3c55d927ca2d admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:12.150579 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432428e9-7d57-42df-8391-3c55d927ca2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:12.152148 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432428e9-7d57-42df-8391-3c55d927ca2d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:12.162506 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1400/2826] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:57:12.166454 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc759cd-3d75-457e-86e2-9e92b53039a5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:12.176513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc759cd-3d75-457e-86e2-9e92b53039a5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:12.181169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ab68-f5a1-4c38-b790-13bd51026e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:12.182038 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ab68-f5a1-4c38-b790-13bd51026e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:12.182321 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ab68-f5a1-4c38-b790-13bd51026e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:12.182599 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ab68-f5a1-4c38-b790-13bd51026e49 admin admin] PATH_INFO: `/v3/projects/fae0b66dc8b046d0a1e5b9a305abcd9c/users/4f30d9f66dba40b6abe7838ed5ea76cd/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:12.197478 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86ab68-f5a1-4c38-b790-13bd51026e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae0b66dc8b046d0a1e5b9a305abcd9c, user_id=4f30d9f66dba40b6abe7838ed5ea76cd, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:12.200119 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86ab68-f5a1-4c38-b790-13bd51026e49 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:12.213538 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1427/2827] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:57:12 2022] PUT /identity/v3/projects/fae0b66dc8b046d0a1e5b9a305abcd9c/users/4f30d9f66dba40b6abe7838ed5ea76cd/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:57:12.223703 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7492de-5d66-4658-bb02-c505bf2978f8 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:12.223995 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7492de-5d66-4658-bb02-c505bf2978f8 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:12.224276 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7492de-5d66-4658-bb02-c505bf2978f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:12.249944 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-4a7492de-5d66-4658-bb02-c505bf2978f8 None None] MFA Rules not processed for user `4f30d9f66dba40b6abe7838ed5ea76cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:12.251172 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-4a7492de-5d66-4658-bb02-c505bf2978f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:12.315507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1401/2828] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:12.326059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ab68-f5a1-4c38-b790-13bd51026e49 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:12.327676 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3b86ab68-f5a1-4c38-b790-13bd51026e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:12.350546 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:12.388059 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ab68-f5a1-4c38-b790-13bd51026e49 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:12.392411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472cf3b-23bf-4cc0-90a1-c4118036da8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:12.393093 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472cf3b-23bf-4cc0-90a1-c4118036da8b service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:12.393945 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472cf3b-23bf-4cc0-90a1-c4118036da8b service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:12.394219 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472cf3b-23bf-4cc0-90a1-c4118036da8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:12.402464 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472cf3b-23bf-4cc0-90a1-c4118036da8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:12.403890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472cf3b-23bf-4cc0-90a1-c4118036da8b service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:12.412766 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1428/2829] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:57:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:57:22.312128 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:22.408569 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=17 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:30.582782 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7492de-5d66-4658-bb02-c505bf2978f8 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:30.644675 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7492de-5d66-4658-bb02-c505bf2978f8 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:30.649309 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:30.650204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:30.650481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:30.650754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] PATH_INFO: `/v3/users/4f30d9f66dba40b6abe7838ed5ea76cd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:30.651923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f30d9f66dba40b6abe7838ed5ea76cd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:30.653858 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:30.700628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f30d9f66dba40b6abe7838ed5ea76cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:30.712808 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f30d9f66dba40b6abe7838ed5ea76cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:30.720031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f30d9f66dba40b6abe7838ed5ea76cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:30.720716 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f30d9f66dba40b6abe7838ed5ea76cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:30.728330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f30d9f66dba40b6abe7838ed5ea76cd'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:30.734144 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1402/2830] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:57:30 2022] DELETE /identity/v3/users/4f30d9f66dba40b6abe7838ed5ea76cd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:57:30.738610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55097530-3a44-426e-a295-c9a5df921473 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:30.739113 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55097530-3a44-426e-a295-c9a5df921473 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:30.739211 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55097530-3a44-426e-a295-c9a5df921473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:30.757060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-55097530-3a44-426e-a295-c9a5df921473 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:30.758265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-55097530-3a44-426e-a295-c9a5df921473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:30.816769 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1429/2831] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:30.826240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:30.827663 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:30.886898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bd837-a72f-4317-a5f4-7b656259ff2c admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:30.890591 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78976b6-515d-4420-870f-87a8408820dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:30.891240 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78976b6-515d-4420-870f-87a8408820dc service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:30.891526 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78976b6-515d-4420-870f-87a8408820dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:30.892062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78976b6-515d-4420-870f-87a8408820dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:30.902221 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78976b6-515d-4420-870f-87a8408820dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:30.903655 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78976b6-515d-4420-870f-87a8408820dc service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:30.914055 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1403/2832] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:57:31.182169 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55097530-3a44-426e-a295-c9a5df921473 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:31.183692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-55097530-3a44-426e-a295-c9a5df921473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:31.196654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55097530-3a44-426e-a295-c9a5df921473 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:31.201922 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:31.202133 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.202412 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.202692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] PATH_INFO: `/v3/projects/fae0b66dc8b046d0a1e5b9a305abcd9c` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.203897 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae0b66dc8b046d0a1e5b9a305abcd9c)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:31.205899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:31.252890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae0b66dc8b046d0a1e5b9a305abcd9c', 'request_id': 'req-1875806c-09db-4172-a52d-e3381842cf5e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:31.261800 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae0b66dc8b046d0a1e5b9a305abcd9c', 'request_id': 'req-1875806c-09db-4172-a52d-e3381842cf5e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:31.268451 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae0b66dc8b046d0a1e5b9a305abcd9c', 'request_id': 'req-1875806c-09db-4172-a52d-e3381842cf5e'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:31.279588 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] The token cache is being invalidate because project fae0b66dc8b046d0a1e5b9a305abcd9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:57:31.279879 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:31.280511 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-1875806c-09db-4172-a52d-e3381842cf5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:57:31.282070 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1430/2833] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:57:31 2022] DELETE /identity/v3/projects/fae0b66dc8b046d0a1e5b9a305abcd9c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:57:31.287643 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929fad0-80b6-4948-87dc-07095cf5ab15 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.287813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929fad0-80b6-4948-87dc-07095cf5ab15 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.288088 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929fad0-80b6-4948-87dc-07095cf5ab15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.306031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-6929fad0-80b6-4948-87dc-07095cf5ab15 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:31.307210 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6929fad0-80b6-4948-87dc-07095cf5ab15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:31.323500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:31.368670 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1404/2834] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:31.376626 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94795219-b2d2-4843-b448-4aa95e3ead60 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.376925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94795219-b2d2-4843-b448-4aa95e3ead60 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.377239 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94795219-b2d2-4843-b448-4aa95e3ead60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.397288 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-94795219-b2d2-4843-b448-4aa95e3ead60 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:31.398501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-94795219-b2d2-4843-b448-4aa95e3ead60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:31.406320 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1431/2835] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:31.410491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929fad0-80b6-4948-87dc-07095cf5ab15 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:31.420572 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929fad0-80b6-4948-87dc-07095cf5ab15 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:31.425052 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea482ad-3a33-4303-ae2a-bcd5cafe06df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:31.425695 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea482ad-3a33-4303-ae2a-bcd5cafe06df admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.425977 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea482ad-3a33-4303-ae2a-bcd5cafe06df admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.426254 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea482ad-3a33-4303-ae2a-bcd5cafe06df admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.426996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea482ad-3a33-4303-ae2a-bcd5cafe06df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:57:31.427843 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea482ad-3a33-4303-ae2a-bcd5cafe06df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:31.429443 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea482ad-3a33-4303-ae2a-bcd5cafe06df admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:31.440921 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1405/2836] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:57:31.444989 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94795219-b2d2-4843-b448-4aa95e3ead60 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:31.454075 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94795219-b2d2-4843-b448-4aa95e3ead60 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:31.458622 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c687d8-eb16-4cb4-81de-d849ec8e9914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:31.459260 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c687d8-eb16-4cb4-81de-d849ec8e9914 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.459541 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c687d8-eb16-4cb4-81de-d849ec8e9914 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.459812 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c687d8-eb16-4cb4-81de-d849ec8e9914 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.460491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c687d8-eb16-4cb4-81de-d849ec8e9914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:31.462407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c687d8-eb16-4cb4-81de-d849ec8e9914 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:31.492506 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1432/2837] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:57:31 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:57:31.493828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea482ad-3a33-4303-ae2a-bcd5cafe06df admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:31.503903 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea482ad-3a33-4303-ae2a-bcd5cafe06df admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:31.563415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff41eaf-7628-4a50-b015-eb6c88510daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:31.564081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41eaf-7628-4a50-b015-eb6c88510daf admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.564365 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41eaf-7628-4a50-b015-eb6c88510daf admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.564671 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41eaf-7628-4a50-b015-eb6c88510daf admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.565356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff41eaf-7628-4a50-b015-eb6c88510daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:31.567288 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff41eaf-7628-4a50-b015-eb6c88510daf admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:31.587651 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1406/2838] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:57:31 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:57:31.591956 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c687d8-eb16-4cb4-81de-d849ec8e9914 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:31.602969 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c687d8-eb16-4cb4-81de-d849ec8e9914 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:31.607510 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6fe4d-37a5-4444-a512-e96392aadc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:31.608160 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6fe4d-37a5-4444-a512-e96392aadc83 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.608442 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6fe4d-37a5-4444-a512-e96392aadc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.608757 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6fe4d-37a5-4444-a512-e96392aadc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.609476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6fe4d-37a5-4444-a512-e96392aadc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:31.611086 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6fe4d-37a5-4444-a512-e96392aadc83 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:31.621602 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1433/2839] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:57:31.625331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff41eaf-7628-4a50-b015-eb6c88510daf admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:31.635356 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff41eaf-7628-4a50-b015-eb6c88510daf admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:31.640000 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81f06d-0fce-4bda-8b10-b691e80f775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:31.640722 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81f06d-0fce-4bda-8b10-b691e80f775c admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.641006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81f06d-0fce-4bda-8b10-b691e80f775c admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.641282 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81f06d-0fce-4bda-8b10-b691e80f775c admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.642263 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81f06d-0fce-4bda-8b10-b691e80f775c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:31.643827 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81f06d-0fce-4bda-8b10-b691e80f775c admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:31.654184 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1407/2840] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:57:31.658173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6fe4d-37a5-4444-a512-e96392aadc83 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:31.668802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6fe4d-37a5-4444-a512-e96392aadc83 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:31.673463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c2866-7cd5-4d07-bd4a-ea8665fd94b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:31.674382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c2866-7cd5-4d07-bd4a-ea8665fd94b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.674667 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c2866-7cd5-4d07-bd4a-ea8665fd94b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.674966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c2866-7cd5-4d07-bd4a-ea8665fd94b0 admin admin] PATH_INFO: `/v3/projects/cc9f5775196a45df8de41ccff957b6ad/users/a23c541d3c33423597b22a233d899c36/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.690507 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c2866-7cd5-4d07-bd4a-ea8665fd94b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc9f5775196a45df8de41ccff957b6ad, user_id=a23c541d3c33423597b22a233d899c36, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:31.693174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c2866-7cd5-4d07-bd4a-ea8665fd94b0 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:31.706307 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1434/2841] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:57:31 2022] PUT /identity/v3/projects/cc9f5775196a45df8de41ccff957b6ad/users/a23c541d3c33423597b22a233d899c36/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:57:31.712009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da75b-2c37-4cf4-b8d9-05cf7113ec5f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.712297 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da75b-2c37-4cf4-b8d9-05cf7113ec5f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.712615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8da75b-2c37-4cf4-b8d9-05cf7113ec5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.729898 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-7a8da75b-2c37-4cf4-b8d9-05cf7113ec5f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:31.731081 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-7a8da75b-2c37-4cf4-b8d9-05cf7113ec5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:31.790834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1408/2842] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:31.800332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c2866-7cd5-4d07-bd4a-ea8665fd94b0 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:31.802111 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5f3c2866-7cd5-4d07-bd4a-ea8665fd94b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:31.824179 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:31.861175 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c2866-7cd5-4d07-bd4a-ea8665fd94b0 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:31.865435 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10baa4bf-88f2-4ac3-bf57-c8e23924e4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:31.866089 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10baa4bf-88f2-4ac3-bf57-c8e23924e4cb service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:31.866377 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10baa4bf-88f2-4ac3-bf57-c8e23924e4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:31.866661 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10baa4bf-88f2-4ac3-bf57-c8e23924e4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:31.875718 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10baa4bf-88f2-4ac3-bf57-c8e23924e4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:31.877143 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10baa4bf-88f2-4ac3-bf57-c8e23924e4cb service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:31.887184 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1435/2843] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:57:35.893960 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36103d-79c1-402c-95cc-7f0a3a19a9c9 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:35.894500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36103d-79c1-402c-95cc-7f0a3a19a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:35.894500 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36103d-79c1-402c-95cc-7f0a3a19a9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:35.923740 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-1e36103d-79c1-402c-95cc-7f0a3a19a9c9 None None] MFA Rules not processed for user `a23c541d3c33423597b22a233d899c36`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:35.925075 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-1e36103d-79c1-402c-95cc-7f0a3a19a9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:35.944208 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:35.996785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1409/2844] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:36.001306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10baa4bf-88f2-4ac3-bf57-c8e23924e4cb service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:36.012703 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10baa4bf-88f2-4ac3-bf57-c8e23924e4cb service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:36.017641 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f11c11-a1fd-496f-8dfb-d1c426f5b52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:36.018330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f11c11-a1fd-496f-8dfb-d1c426f5b52c admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:36.018612 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f11c11-a1fd-496f-8dfb-d1c426f5b52c admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:36.018887 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f11c11-a1fd-496f-8dfb-d1c426f5b52c admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:36.019593 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f11c11-a1fd-496f-8dfb-d1c426f5b52c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:36.022721 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f11c11-a1fd-496f-8dfb-d1c426f5b52c admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:36.048228 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1436/2845] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:57:36 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:57:36.052321 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36103d-79c1-402c-95cc-7f0a3a19a9c9 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:36.063242 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36103d-79c1-402c-95cc-7f0a3a19a9c9 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:36.120352 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf68f29-a927-40db-9cae-40668d9aa315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:36.120982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf68f29-a927-40db-9cae-40668d9aa315 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:36.121265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf68f29-a927-40db-9cae-40668d9aa315 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:36.121551 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf68f29-a927-40db-9cae-40668d9aa315 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:36.122237 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf68f29-a927-40db-9cae-40668d9aa315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:36.124143 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf68f29-a927-40db-9cae-40668d9aa315 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:36.152118 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1410/2846] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:57:36 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:57:36.153379 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f11c11-a1fd-496f-8dfb-d1c426f5b52c admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:36.164986 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f11c11-a1fd-496f-8dfb-d1c426f5b52c admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:36.170018 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efde260-7322-452a-be5b-1cb1875876a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:36.170694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efde260-7322-452a-be5b-1cb1875876a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:36.170978 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efde260-7322-452a-be5b-1cb1875876a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:36.171255 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efde260-7322-452a-be5b-1cb1875876a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:36.171983 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efde260-7322-452a-be5b-1cb1875876a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:36.173654 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efde260-7322-452a-be5b-1cb1875876a5 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:36.186473 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1437/2847] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:57:36.190891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf68f29-a927-40db-9cae-40668d9aa315 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:36.203053 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf68f29-a927-40db-9cae-40668d9aa315 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:36.208866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53dbc95-8e9d-4945-b6e6-eea64c92f72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:36.209800 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dbc95-8e9d-4945-b6e6-eea64c92f72d admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:36.210080 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dbc95-8e9d-4945-b6e6-eea64c92f72d admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:36.210359 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dbc95-8e9d-4945-b6e6-eea64c92f72d admin admin] PATH_INFO: `/v3/projects/eb03d52c858c4e03bdb435c2c033606a/users/4fc65cc88c3e4d9297069b6b8a0d38d9/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:36.227953 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53dbc95-8e9d-4945-b6e6-eea64c92f72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb03d52c858c4e03bdb435c2c033606a, user_id=4fc65cc88c3e4d9297069b6b8a0d38d9, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:36.230654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53dbc95-8e9d-4945-b6e6-eea64c92f72d admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:36.245241 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1411/2848] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:57:36 2022] PUT /identity/v3/projects/eb03d52c858c4e03bdb435c2c033606a/users/4fc65cc88c3e4d9297069b6b8a0d38d9/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:57:36.249173 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efde260-7322-452a-be5b-1cb1875876a5 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:36.260098 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efde260-7322-452a-be5b-1cb1875876a5 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:36.322675 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318660e-f0c7-49a4-9e11-be1c7b24e8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:36.323360 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318660e-f0c7-49a4-9e11-be1c7b24e8dc admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:36.323663 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318660e-f0c7-49a4-9e11-be1c7b24e8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:36.323928 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318660e-f0c7-49a4-9e11-be1c7b24e8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:36.324696 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318660e-f0c7-49a4-9e11-be1c7b24e8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:36.326673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318660e-f0c7-49a4-9e11-be1c7b24e8dc admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:36.338206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1438/2849] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:57:36.342324 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53dbc95-8e9d-4945-b6e6-eea64c92f72d admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:36.354557 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53dbc95-8e9d-4945-b6e6-eea64c92f72d admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:36.359550 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76c637-016b-4d69-a2f3-7bb9b5042eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:36.360441 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76c637-016b-4d69-a2f3-7bb9b5042eab admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:36.360769 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76c637-016b-4d69-a2f3-7bb9b5042eab admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:36.361046 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76c637-016b-4d69-a2f3-7bb9b5042eab admin admin] PATH_INFO: `/v3/projects/eb03d52c858c4e03bdb435c2c033606a/users/4fc65cc88c3e4d9297069b6b8a0d38d9/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:36.362998 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76c637-016b-4d69-a2f3-7bb9b5042eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb03d52c858c4e03bdb435c2c033606a, user_id=4fc65cc88c3e4d9297069b6b8a0d38d9, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:36.365750 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76c637-016b-4d69-a2f3-7bb9b5042eab admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:36.376164 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1412/2850] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:57:36 2022] PUT /identity/v3/projects/eb03d52c858c4e03bdb435c2c033606a/users/4fc65cc88c3e4d9297069b6b8a0d38d9/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:57:41.024572 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38351a-0a6c-4bd4-84ee-28ddb41d610d None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:41.025151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38351a-0a6c-4bd4-84ee-28ddb41d610d None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:41.025151 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38351a-0a6c-4bd4-84ee-28ddb41d610d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:41.055456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-1f38351a-0a6c-4bd4-84ee-28ddb41d610d None None] MFA Rules not processed for user `4fc65cc88c3e4d9297069b6b8a0d38d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:41.056807 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-1f38351a-0a6c-4bd4-84ee-28ddb41d610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:41.129520 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1439/2851] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:41.143710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76c637-016b-4d69-a2f3-7bb9b5042eab admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:41.151332 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9b76c637-016b-4d69-a2f3-7bb9b5042eab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:41.218656 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76c637-016b-4d69-a2f3-7bb9b5042eab admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:41.222247 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246012b-c6ef-4195-be8c-377a0c370232 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71e5ef9dc32f494791ba4addb5151232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:41.222907 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246012b-c6ef-4195-be8c-377a0c370232 service glance] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:41.223198 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246012b-c6ef-4195-be8c-377a0c370232 service glance] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:41.224006 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246012b-c6ef-4195-be8c-377a0c370232 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:41.250649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:41.285547 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246012b-c6ef-4195-be8c-377a0c370232 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:41.287737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246012b-c6ef-4195-be8c-377a0c370232 service glance] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:41.297520 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1413/2852] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:57:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:57:41.402685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6ac52-3018-4fbd-a03b-698e5d635bd5 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:41.402982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6ac52-3018-4fbd-a03b-698e5d635bd5 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:41.403263 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6ac52-3018-4fbd-a03b-698e5d635bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:41.422403 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3ee6ac52-3018-4fbd-a03b-698e5d635bd5 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:41.423607 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3ee6ac52-3018-4fbd-a03b-698e5d635bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:41.429465 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:41.487097 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1440/2853] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:57:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:41.533976 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246012b-c6ef-4195-be8c-377a0c370232 service glance] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:41.535394 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-0246012b-c6ef-4195-be8c-377a0c370232 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:41.604031 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246012b-c6ef-4195-be8c-377a0c370232 service glance] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:41.609023 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29e432-5fb1-484f-b4a1-eb3b6a5ba349 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:41.609668 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29e432-5fb1-484f-b4a1-eb3b6a5ba349 service swift] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:41.609945 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29e432-5fb1-484f-b4a1-eb3b6a5ba349 service swift] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:41.610220 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29e432-5fb1-484f-b4a1-eb3b6a5ba349 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:41.619117 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29e432-5fb1-484f-b4a1-eb3b6a5ba349 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:41.620856 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29e432-5fb1-484f-b4a1-eb3b6a5ba349 service swift] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:41.629468 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1414/2854] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:57:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:57:50.302187 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086c84b-953a-491c-b78a-bf6038adde48 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:50.302729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086c84b-953a-491c-b78a-bf6038adde48 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:50.302729 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086c84b-953a-491c-b78a-bf6038adde48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:50.321137 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-0086c84b-953a-491c-b78a-bf6038adde48 None None] MFA Rules not processed for user `a23c541d3c33423597b22a233d899c36`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:50.322332 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-0086c84b-953a-491c-b78a-bf6038adde48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:50.329356 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1441/2855] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:57:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:51.625909 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=48 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:57:52.144058 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef81492-93b0-4eef-ae3b-344588e25a20 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:52.144391 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef81492-93b0-4eef-ae3b-344588e25a20 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:52.144717 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef81492-93b0-4eef-ae3b-344588e25a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:52.162380 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-eef81492-93b0-4eef-ae3b-344588e25a20 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:57:52.163645 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-eef81492-93b0-4eef-ae3b-344588e25a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:57:52.171171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1415/2856] 10.222.0.47 () {58 vars in 1009 bytes} [Fri Jun 24 23:57:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:57:52.183708 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086c84b-953a-491c-b78a-bf6038adde48 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:57:52.197047 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086c84b-953a-491c-b78a-bf6038adde48 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:57:52.202241 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5184d2-8398-40df-aa86-a33cca98364b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:57:52.202958 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5184d2-8398-40df-aa86-a33cca98364b service swift] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:57:52.203331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5184d2-8398-40df-aa86-a33cca98364b service swift] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:57:52.203679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5184d2-8398-40df-aa86-a33cca98364b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:57:52.212804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5184d2-8398-40df-aa86-a33cca98364b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:57:52.214504 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5184d2-8398-40df-aa86-a33cca98364b service swift] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:57:52.223448 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1442/2857] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:57:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:58:02.157665 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:58:02.219369 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=29 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:58:16.313108 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef81492-93b0-4eef-ae3b-344588e25a20 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:58:16.368149 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef81492-93b0-4eef-ae3b-344588e25a20 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:58:16.371803 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fe56f-c2e5-4a53-aafb-c4e50bcf8a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:58:16.372384 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fe56f-c2e5-4a53-aafb-c4e50bcf8a6a service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:58:16.372697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fe56f-c2e5-4a53-aafb-c4e50bcf8a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:58:16.372975 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fe56f-c2e5-4a53-aafb-c4e50bcf8a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:58:16.382048 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fe56f-c2e5-4a53-aafb-c4e50bcf8a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:58:16.383455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fe56f-c2e5-4a53-aafb-c4e50bcf8a6a service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:58:16.393122 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1416/2858] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:58:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:58:26.388588 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=26 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:58:42.812802 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5184d2-8398-40df-aa86-a33cca98364b service swift] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:58:42.822831 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5184d2-8398-40df-aa86-a33cca98364b service swift] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:58:42.826482 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:58:42.827125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:58:42.827409 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:58:42.827693 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:58:42.829501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:58:42.909443 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:58:42.910884 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:58:42.923382 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1443/2859] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:58:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:58:44.221043 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fe56f-c2e5-4a53-aafb-c4e50bcf8a6a service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:58:44.222488 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-b59fe56f-c2e5-4a53-aafb-c4e50bcf8a6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:58:44.233767 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fe56f-c2e5-4a53-aafb-c4e50bcf8a6a service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:58:44.237435 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1327c-dfcb-46f1-913f-4a94317a0bda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:58:44.238068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1327c-dfcb-46f1-913f-4a94317a0bda service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:58:44.238357 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1327c-dfcb-46f1-913f-4a94317a0bda service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:58:44.238637 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1327c-dfcb-46f1-913f-4a94317a0bda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:58:44.240371 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f4e1327c-dfcb-46f1-913f-4a94317a0bda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:58:44.254644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1327c-dfcb-46f1-913f-4a94317a0bda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:58:44.256265 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1327c-dfcb-46f1-913f-4a94317a0bda service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:58:44.267819 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1417/2860] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:58:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:58:52.917772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=34 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:58:54.262694 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=6 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:59:27.861707 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:27.866491 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:27.897214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbe838-a9a2-4544-82b0-68471e5c97f3 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:27.908692 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9b6e8-8bbe-48a4-aa97-666f8fa8c8ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:27.910784 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9b6e8-8bbe-48a4-aa97-666f8fa8c8ec service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:27.915685 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9b6e8-8bbe-48a4-aa97-666f8fa8c8ec service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:27.916050 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9b6e8-8bbe-48a4-aa97-666f8fa8c8ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:27.918125 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-9ec9b6e8-8bbe-48a4-aa97-666f8fa8c8ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:27.933640 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9b6e8-8bbe-48a4-aa97-666f8fa8c8ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:27.935610 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9b6e8-8bbe-48a4-aa97-666f8fa8c8ec service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:27.950562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1444/2861] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:59:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:59:28.028900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1327c-dfcb-46f1-913f-4a94317a0bda service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:28.038895 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1327c-dfcb-46f1-913f-4a94317a0bda service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:28.043802 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29118035-6fc5-4813-b74a-5d055bb81fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:28.044604 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29118035-6fc5-4813-b74a-5d055bb81fab service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:28.045068 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29118035-6fc5-4813-b74a-5d055bb81fab service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:28.046126 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29118035-6fc5-4813-b74a-5d055bb81fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:28.048202 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-29118035-6fc5-4813-b74a-5d055bb81fab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:28.062770 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29118035-6fc5-4813-b74a-5d055bb81fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:28.064331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29118035-6fc5-4813-b74a-5d055bb81fab service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:28.075792 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1418/2862] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:59:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:59:34.247417 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9b6e8-8bbe-48a4-aa97-666f8fa8c8ec service nova] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:34.263537 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9b6e8-8bbe-48a4-aa97-666f8fa8c8ec service nova] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:34.271347 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6f59c-7e98-46cf-a0d8-61d277c4e25d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:34.272328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6f59c-7e98-46cf-a0d8-61d277c4e25d service nova] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:34.272658 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6f59c-7e98-46cf-a0d8-61d277c4e25d service nova] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:34.273128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6f59c-7e98-46cf-a0d8-61d277c4e25d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:34.275363 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-17b6f59c-7e98-46cf-a0d8-61d277c4e25d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:34.290673 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6f59c-7e98-46cf-a0d8-61d277c4e25d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:34.292279 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6f59c-7e98-46cf-a0d8-61d277c4e25d service nova] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:34.307573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1445/2863] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:59:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:59:35.404649 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55213e9-16c5-4b96-8155-c628d34b5551 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:35.405162 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55213e9-16c5-4b96-8155-c628d34b5551 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:35.405279 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55213e9-16c5-4b96-8155-c628d34b5551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:35.423423 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-f55213e9-16c5-4b96-8155-c628d34b5551 None None] MFA Rules not processed for user `063b3fbafe0949d1b09bd9b7b0d70131`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:59:35.424636 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f55213e9-16c5-4b96-8155-c628d34b5551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:35.431877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1419/2864] 10.222.0.47 () {58 vars in 967 bytes} [Fri Jun 24 23:59:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:59:35.470977 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6f59c-7e98-46cf-a0d8-61d277c4e25d service nova] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:35.482297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6f59c-7e98-46cf-a0d8-61d277c4e25d service nova] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:35.487177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6921943-29ad-4fd5-b916-76be8dae4e3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dd1e02af2374224b968c2644dd0c813', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:35.487808 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6921943-29ad-4fd5-b916-76be8dae4e3a service swift] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:35.488839 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6921943-29ad-4fd5-b916-76be8dae4e3a service swift] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:35.489001 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6921943-29ad-4fd5-b916-76be8dae4e3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:35.497065 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6921943-29ad-4fd5-b916-76be8dae4e3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:35.498637 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6921943-29ad-4fd5-b916-76be8dae4e3a service swift] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:35.507088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1446/2865] 10.222.0.47 () {60 vars in 1384 bytes} [Fri Jun 24 23:59:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 23:59:42.107217 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55213e9-16c5-4b96-8155-c628d34b5551 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:42.172302 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55213e9-16c5-4b96-8155-c628d34b5551 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:42.176810 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:42.178063 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:42.178351 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:42.178629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] PATH_INFO: `/v3/users/a23c541d3c33423597b22a233d899c36` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:42.179836 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23c541d3c33423597b22a233d899c36)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:42.182101 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:42.233820 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23c541d3c33423597b22a233d899c36'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.246171 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23c541d3c33423597b22a233d899c36'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.254268 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23c541d3c33423597b22a233d899c36'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.254957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23c541d3c33423597b22a233d899c36'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.262546 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23c541d3c33423597b22a233d899c36'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.270429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1420/2866] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:59:42 2022] DELETE /identity/v3/users/a23c541d3c33423597b22a233d899c36 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:59:42.464498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6921943-29ad-4fd5-b916-76be8dae4e3a service swift] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:42.476102 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6921943-29ad-4fd5-b916-76be8dae4e3a service swift] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:42.535174 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:42.536226 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:42.536513 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:42.536827 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] PATH_INFO: `/v3/projects/cc9f5775196a45df8de41ccff957b6ad` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:42.538029 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc9f5775196a45df8de41ccff957b6ad)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:42.539933 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:42.585212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc9f5775196a45df8de41ccff957b6ad', 'request_id': 'req-35980dff-222e-4d06-bc32-1404d6d78f27'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.591832 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc9f5775196a45df8de41ccff957b6ad', 'request_id': 'req-35980dff-222e-4d06-bc32-1404d6d78f27'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.599063 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc9f5775196a45df8de41ccff957b6ad', 'request_id': 'req-35980dff-222e-4d06-bc32-1404d6d78f27'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.609331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] The token cache is being invalidate because project cc9f5775196a45df8de41ccff957b6ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:59:42.609616 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.610267 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.611999 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1447/2867] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:59:42 2022] DELETE /identity/v3/projects/cc9f5775196a45df8de41ccff957b6ad => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:59:42.616573 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:42.618056 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:42.631677 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f5932-f60e-49df-bc08-6fc51b8270bc admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:42.687510 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:42.688382 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:42.688704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:42.688989 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] PATH_INFO: `/v3/users/4fc65cc88c3e4d9297069b6b8a0d38d9` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:42.690204 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc65cc88c3e4d9297069b6b8a0d38d9)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:42.692353 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:42.693847 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:59:42.741301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc65cc88c3e4d9297069b6b8a0d38d9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.752923 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc65cc88c3e4d9297069b6b8a0d38d9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.760835 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc65cc88c3e4d9297069b6b8a0d38d9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.761661 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc65cc88c3e4d9297069b6b8a0d38d9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.769598 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-ed9bd837-05b6-4788-91f8-a6876c62b743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc65cc88c3e4d9297069b6b8a0d38d9'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:42.775710 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1421/2868] 10.222.0.47 () {58 vars in 1223 bytes} [Fri Jun 24 23:59:42 2022] DELETE /identity/v3/users/4fc65cc88c3e4d9297069b6b8a0d38d9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:59:42.949690 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:42.951310 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:42.966301 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35980dff-222e-4d06-bc32-1404d6d78f27 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:43.017183 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:59:43.028900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:43.029775 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.030060 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.030333 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] PATH_INFO: `/v3/projects/eb03d52c858c4e03bdb435c2c033606a` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.031498 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb03d52c858c4e03bdb435c2c033606a)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:43.033499 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:43.078946 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb03d52c858c4e03bdb435c2c033606a', 'request_id': 'req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:43.085902 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb03d52c858c4e03bdb435c2c033606a', 'request_id': 'req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:43.091815 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb03d52c858c4e03bdb435c2c033606a', 'request_id': 'req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:43.101844 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] The token cache is being invalidate because project eb03d52c858c4e03bdb435c2c033606a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 23:59:43.102128 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:43.102753 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-8dd68bb0-aa83-4fc4-985f-bde5bdf44b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 23:59:43.104272 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1448/2869] 10.222.0.47 () {58 vars in 1232 bytes} [Fri Jun 24 23:59:42 2022] DELETE /identity/v3/projects/eb03d52c858c4e03bdb435c2c033606a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:59:43.109434 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ccd54-96b3-4c94-b499-46479972b28f None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.109924 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ccd54-96b3-4c94-b499-46479972b28f None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.110106 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ccd54-96b3-4c94-b499-46479972b28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.128444 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-456ccd54-96b3-4c94-b499-46479972b28f None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:59:43.130742 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-456ccd54-96b3-4c94-b499-46479972b28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:43.189762 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1422/2870] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:59:43.197918 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d785a96-aeeb-4921-8753-f4e550581717 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.198214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d785a96-aeeb-4921-8753-f4e550581717 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.198493 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d785a96-aeeb-4921-8753-f4e550581717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.217242 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-2d785a96-aeeb-4921-8753-f4e550581717 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:59:43.218628 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-2d785a96-aeeb-4921-8753-f4e550581717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:43.228428 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1449/2871] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:59:43.232707 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ccd54-96b3-4c94-b499-46479972b28f None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:43.243007 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ccd54-96b3-4c94-b499-46479972b28f None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:43.247524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d4d67-d9f7-4b59-8a69-9aecc0d293e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:43.248166 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d4d67-d9f7-4b59-8a69-9aecc0d293e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.248449 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d4d67-d9f7-4b59-8a69-9aecc0d293e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.248760 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d4d67-d9f7-4b59-8a69-9aecc0d293e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.249481 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d4d67-d9f7-4b59-8a69-9aecc0d293e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 23:59:43.249813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d4d67-d9f7-4b59-8a69-9aecc0d293e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:43.251376 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d4d67-d9f7-4b59-8a69-9aecc0d293e6 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:43.263109 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1423/2872] 10.222.0.47 () {58 vars in 1152 bytes} [Fri Jun 24 23:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 23:59:43.267688 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d785a96-aeeb-4921-8753-f4e550581717 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:43.276991 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d785a96-aeeb-4921-8753-f4e550581717 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:43.281689 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65940836-fbab-4565-87d6-cf7ee6f2ad22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:43.282330 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65940836-fbab-4565-87d6-cf7ee6f2ad22 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.282613 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65940836-fbab-4565-87d6-cf7ee6f2ad22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.282888 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65940836-fbab-4565-87d6-cf7ee6f2ad22 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.283573 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65940836-fbab-4565-87d6-cf7ee6f2ad22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:43.286073 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65940836-fbab-4565-87d6-cf7ee6f2ad22 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:43.315414 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1450/2873] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:59:43 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:59:43.316736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d4d67-d9f7-4b59-8a69-9aecc0d293e6 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:43.326915 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d4d67-d9f7-4b59-8a69-9aecc0d293e6 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:43.381389 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c0a79-b41a-40c4-a7ff-93d18f269eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:43.382047 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0a79-b41a-40c4-a7ff-93d18f269eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.382330 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0a79-b41a-40c4-a7ff-93d18f269eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.382609 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0a79-b41a-40c4-a7ff-93d18f269eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.383300 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c0a79-b41a-40c4-a7ff-93d18f269eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:43.385228 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c0a79-b41a-40c4-a7ff-93d18f269eb9 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:43.394849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:59:43.409044 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1424/2874] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:59:43 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 23:59:43.410864 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65940836-fbab-4565-87d6-cf7ee6f2ad22 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:43.421291 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65940836-fbab-4565-87d6-cf7ee6f2ad22 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:43.427620 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1112ac9-617f-4141-bd4f-cd7ba9f1e8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:43.428411 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1112ac9-617f-4141-bd4f-cd7ba9f1e8aa admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.428849 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1112ac9-617f-4141-bd4f-cd7ba9f1e8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.429214 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1112ac9-617f-4141-bd4f-cd7ba9f1e8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.430002 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1112ac9-617f-4141-bd4f-cd7ba9f1e8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:43.431657 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1112ac9-617f-4141-bd4f-cd7ba9f1e8aa admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:43.442328 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1451/2875] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:59:43.446264 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c0a79-b41a-40c4-a7ff-93d18f269eb9 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:43.456283 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c0a79-b41a-40c4-a7ff-93d18f269eb9 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:43.460893 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a2b40-858d-4efe-8cf7-7c8bb011fe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:43.461542 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a2b40-858d-4efe-8cf7-7c8bb011fe94 admin admin] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.461825 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a2b40-858d-4efe-8cf7-7c8bb011fe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.462102 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a2b40-858d-4efe-8cf7-7c8bb011fe94 admin admin] PATH_INFO: `/v3/roles` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.462824 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a2b40-858d-4efe-8cf7-7c8bb011fe94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:43.464429 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a2b40-858d-4efe-8cf7-7c8bb011fe94 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:43.475019 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1425/2876] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 23:59:43.479262 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1112ac9-617f-4141-bd4f-cd7ba9f1e8aa admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:43.489890 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1112ac9-617f-4141-bd4f-cd7ba9f1e8aa admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:43.494542 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b6a4d-b5bb-491a-923f-ccb33682b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:43.495416 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b6a4d-b5bb-491a-923f-ccb33682b9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.495701 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b6a4d-b5bb-491a-923f-ccb33682b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.495982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b6a4d-b5bb-491a-923f-ccb33682b9d7 admin admin] PATH_INFO: `/v3/projects/f10b8a1bc1654ae4ac5c08c23ca9e399/users/bd94685199f44a218dd96397b02cdc39/roles/86c49ec8587244e48fab81862ddb3641` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.510250 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b6a4d-b5bb-491a-923f-ccb33682b9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10b8a1bc1654ae4ac5c08c23ca9e399, user_id=bd94685199f44a218dd96397b02cdc39, role_id=86c49ec8587244e48fab81862ddb3641)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:43.512929 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b6a4d-b5bb-491a-923f-ccb33682b9d7 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:43.527088 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1452/2877] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:59:43 2022] PUT /identity/v3/projects/f10b8a1bc1654ae4ac5c08c23ca9e399/users/bd94685199f44a218dd96397b02cdc39/roles/86c49ec8587244e48fab81862ddb3641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 23:59:43.532133 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ed9ba-b29e-4f07-abd7-a83ff9d064c2 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.532428 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ed9ba-b29e-4f07-abd7-a83ff9d064c2 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.532748 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ed9ba-b29e-4f07-abd7-a83ff9d064c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.551795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-9e8ed9ba-b29e-4f07-abd7-a83ff9d064c2 None None] MFA Rules not processed for user `19e6b8d9d8f94b0c8f316b8b2060e76d`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:59:43.553032 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-9e8ed9ba-b29e-4f07-abd7-a83ff9d064c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:43.614431 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1426/2878] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:59:43.624165 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b6a4d-b5bb-491a-923f-ccb33682b9d7 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:43.625880 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-948b6a4d-b5bb-491a-923f-ccb33682b9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:43.687122 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b6a4d-b5bb-491a-923f-ccb33682b9d7 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:43.691049 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9662f-e70b-4d3a-8ccd-2ee9300f2181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:43.691714 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9662f-e70b-4d3a-8ccd-2ee9300f2181 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:43.692017 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9662f-e70b-4d3a-8ccd-2ee9300f2181 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:43.692297 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9662f-e70b-4d3a-8ccd-2ee9300f2181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:43.701452 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9662f-e70b-4d3a-8ccd-2ee9300f2181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:43.702833 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9662f-e70b-4d3a-8ccd-2ee9300f2181 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:43.712925 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1453/2879] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 23:59:47.902795 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23740-cb17-4aae-ab4e-81544c0ffc55 None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:47.903578 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23740-cb17-4aae-ab4e-81544c0ffc55 None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:47.904168 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23740-cb17-4aae-ab4e-81544c0ffc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:47.938524 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-96d23740-cb17-4aae-ab4e-81544c0ffc55 None None] MFA Rules not processed for user `bd94685199f44a218dd96397b02cdc39`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:59:47.940038 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-96d23740-cb17-4aae-ab4e-81544c0ffc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:48.017175 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1427/2880] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:59:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:59:48.021229 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9662f-e70b-4d3a-8ccd-2ee9300f2181 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:48.031235 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9662f-e70b-4d3a-8ccd-2ee9300f2181 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:48.036678 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca9cb8-29dd-45b1-9320-1131074d0028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:48.037345 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca9cb8-29dd-45b1-9320-1131074d0028 admin admin] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:48.037634 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca9cb8-29dd-45b1-9320-1131074d0028 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:48.037913 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca9cb8-29dd-45b1-9320-1131074d0028 admin admin] PATH_INFO: `/v3/projects` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:48.038605 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca9cb8-29dd-45b1-9320-1131074d0028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:48.040528 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca9cb8-29dd-45b1-9320-1131074d0028 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:48.062275 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:59:48.074950 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1454/2881] 10.222.0.47 () {60 vars in 1152 bytes} [Fri Jun 24 23:59:48 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:59:48.076853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23740-cb17-4aae-ab4e-81544c0ffc55 None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:48.088849 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23740-cb17-4aae-ab4e-81544c0ffc55 None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:48.147628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653f939-ee4c-41cf-82f1-406eb563da33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:48.148337 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653f939-ee4c-41cf-82f1-406eb563da33 admin admin] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:48.148630 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653f939-ee4c-41cf-82f1-406eb563da33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:48.148917 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653f939-ee4c-41cf-82f1-406eb563da33 admin admin] PATH_INFO: `/v3/users` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:48.149622 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653f939-ee4c-41cf-82f1-406eb563da33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:48.151577 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653f939-ee4c-41cf-82f1-406eb563da33 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:48.173086 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1428/2882] 10.222.0.47 () {60 vars in 1143 bytes} [Fri Jun 24 23:59:48 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 23:59:48.177193 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca9cb8-29dd-45b1-9320-1131074d0028 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:48.189006 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca9cb8-29dd-45b1-9320-1131074d0028 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:48.193748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2a2a3-8093-4242-8824-e741da97c775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:48.194430 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2a2a3-8093-4242-8824-e741da97c775 admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:48.194712 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2a2a3-8093-4242-8824-e741da97c775 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:48.194990 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2a2a3-8093-4242-8824-e741da97c775 admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:48.195716 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2a2a3-8093-4242-8824-e741da97c775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:48.197365 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2a2a3-8093-4242-8824-e741da97c775 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:48.209085 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1455/2883] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:59:48.213314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653f939-ee4c-41cf-82f1-406eb563da33 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:48.223454 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 23:59:48.229396 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653f939-ee4c-41cf-82f1-406eb563da33 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:48.240158 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b38111-a793-488c-adf9-e3e2316ca3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:48.240158 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b38111-a793-488c-adf9-e3e2316ca3d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:48.240455 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b38111-a793-488c-adf9-e3e2316ca3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:48.240850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b38111-a793-488c-adf9-e3e2316ca3d1 admin admin] PATH_INFO: `/v3/projects/4a9c7101421e47999b953bfa8e08d5a3/users/7c980082abfe4710ab6a21cd0d827808/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:48.257654 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b38111-a793-488c-adf9-e3e2316ca3d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9c7101421e47999b953bfa8e08d5a3, user_id=7c980082abfe4710ab6a21cd0d827808, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:48.262854 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b38111-a793-488c-adf9-e3e2316ca3d1 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:48.281107 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1429/2884] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:59:48 2022] PUT /identity/v3/projects/4a9c7101421e47999b953bfa8e08d5a3/users/7c980082abfe4710ab6a21cd0d827808/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:59:48.282565 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2a2a3-8093-4242-8824-e741da97c775 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:48.294221 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2a2a3-8093-4242-8824-e741da97c775 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:48.357501 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2acde-1cd2-4984-83f6-340f39831e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:48.357679 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2acde-1cd2-4984-83f6-340f39831e2a admin admin] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:48.357936 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2acde-1cd2-4984-83f6-340f39831e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:48.358215 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2acde-1cd2-4984-83f6-340f39831e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:48.358943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2acde-1cd2-4984-83f6-340f39831e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:48.360552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2acde-1cd2-4984-83f6-340f39831e2a admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:48.370704 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1456/2885] 10.222.0.47 () {58 vars in 1121 bytes} [Fri Jun 24 23:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 23:59:48.374764 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b38111-a793-488c-adf9-e3e2316ca3d1 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:48.385982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b38111-a793-488c-adf9-e3e2316ca3d1 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:48.390629 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13909078-84cb-4804-8b99-675cef998f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:48.391505 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13909078-84cb-4804-8b99-675cef998f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:48.392036 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13909078-84cb-4804-8b99-675cef998f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:48.392314 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13909078-84cb-4804-8b99-675cef998f94 admin admin] PATH_INFO: `/v3/projects/4a9c7101421e47999b953bfa8e08d5a3/users/7c980082abfe4710ab6a21cd0d827808/roles/419ec418f6544c938d3669a80f7b2ffd` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:48.394502 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13909078-84cb-4804-8b99-675cef998f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a9c7101421e47999b953bfa8e08d5a3, user_id=7c980082abfe4710ab6a21cd0d827808, role_id=419ec418f6544c938d3669a80f7b2ffd)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:48.397157 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13909078-84cb-4804-8b99-675cef998f94 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:48.407650 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1430/2886] 10.222.0.47 () {60 vars in 1482 bytes} [Fri Jun 24 23:59:48 2022] PUT /identity/v3/projects/4a9c7101421e47999b953bfa8e08d5a3/users/7c980082abfe4710ab6a21cd0d827808/roles/419ec418f6544c938d3669a80f7b2ffd => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 23:59:53.065166 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e832ae-95ef-4a4f-8cfb-d7055750e958 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:53.065697 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e832ae-95ef-4a4f-8cfb-d7055750e958 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:53.065810 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e832ae-95ef-4a4f-8cfb-d7055750e958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:53.097648 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-30e832ae-95ef-4a4f-8cfb-d7055750e958 None None] MFA Rules not processed for user `7c980082abfe4710ab6a21cd0d827808`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 23:59:53.098943 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-30e832ae-95ef-4a4f-8cfb-d7055750e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:53.183023 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1457/2887] 10.222.0.47 () {58 vars in 957 bytes} [Fri Jun 24 23:59:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 23:59:53.198422 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13909078-84cb-4804-8b99-675cef998f94 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 23:59:53.200415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-13909078-84cb-4804-8b99-675cef998f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 23:59:53.281878 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13909078-84cb-4804-8b99-675cef998f94 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 23:59:53.287877 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d4eb5-e660-40b6-a7f9-23b5b1b14cd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 23:59:53.288534 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d4eb5-e660-40b6-a7f9-23b5b1b14cd6 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 23:59:53.288853 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d4eb5-e660-40b6-a7f9-23b5b1b14cd6 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 23:59:53.289145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d4eb5-e660-40b6-a7f9-23b5b1b14cd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 23:59:53.350233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d4eb5-e660-40b6-a7f9-23b5b1b14cd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 23:59:53.351875 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d4eb5-e660-40b6-a7f9-23b5b1b14cd6 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 23:59:53.361628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1431/2888] 10.222.0.47 () {60 vars in 1365 bytes} [Fri Jun 24 23:59:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:00:03.179858 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=69 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:00:03.357615 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=69,keystone:INSERT=2 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:00:09.812559 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb54692-8179-4ae0-aeda-6995e83ecbfe None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:00:09.813083 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb54692-8179-4ae0-aeda-6995e83ecbfe None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:00:09.813199 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb54692-8179-4ae0-aeda-6995e83ecbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:00:09.830463 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-bfb54692-8179-4ae0-aeda-6995e83ecbfe None None] MFA Rules not processed for user `bd94685199f44a218dd96397b02cdc39`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 00:00:09.831671 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-bfb54692-8179-4ae0-aeda-6995e83ecbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:00:09.839146 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1458/2889] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jun 25 00:00:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 00:00:10.160193 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d4eb5-e660-40b6-a7f9-23b5b1b14cd6 service nova] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:00:10.161745 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-477d4eb5-e660-40b6-a7f9-23b5b1b14cd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:00:10.218913 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d4eb5-e660-40b6-a7f9-23b5b1b14cd6 service nova] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:00:10.222687 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7886-e87b-465f-9523-d15806c6c416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:00:10.223331 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7886-e87b-465f-9523-d15806c6c416 service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:00:10.223616 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7886-e87b-465f-9523-d15806c6c416 service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:00:10.224009 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7886-e87b-465f-9523-d15806c6c416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:00:10.225940 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-ccfd7886-e87b-465f-9523-d15806c6c416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:00:10.235644 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd7886-e87b-465f-9523-d15806c6c416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:00:10.237145 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd7886-e87b-465f-9523-d15806c6c416 service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:00:10.245425 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1432/2890] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:00:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:00:12.703195 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb54692-8179-4ae0-aeda-6995e83ecbfe None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:00:12.704643 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-bfb54692-8179-4ae0-aeda-6995e83ecbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:00:12.781823 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb54692-8179-4ae0-aeda-6995e83ecbfe None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:00:12.790025 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cd966-3773-46cb-a733-ffa2b93f736c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:00:12.793728 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cd966-3773-46cb-a733-ffa2b93f736c service placement] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:00:12.797591 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cd966-3773-46cb-a733-ffa2b93f736c service placement] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:00:12.797875 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1cd966-3773-46cb-a733-ffa2b93f736c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:00:12.809764 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-5a1cd966-3773-46cb-a733-ffa2b93f736c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:00:12.822804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cd966-3773-46cb-a733-ffa2b93f736c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:00:12.824655 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1cd966-3773-46cb-a733-ffa2b93f736c service placement] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:00:12.840021 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1459/2891] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:00:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:00:20.241741 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=26 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:00:22.836217 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=35 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:00:28.109491 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7886-e87b-465f-9523-d15806c6c416 service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:00:28.126470 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7886-e87b-465f-9523-d15806c6c416 service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:00:28.132277 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:00:28.132981 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:00:28.133248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:00:28.133536 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:00:28.135377 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:00:28.149159 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:00:28.150811 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:00:28.162780 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1433/2892] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:00:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:00:30.380650 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cd966-3773-46cb-a733-ffa2b93f736c service placement] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:00:30.390587 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1cd966-3773-46cb-a733-ffa2b93f736c service placement] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:00:30.394397 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31397ddf-e81c-4a48-a056-f49d22e2bb42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:00:30.395026 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31397ddf-e81c-4a48-a056-f49d22e2bb42 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:00:30.395306 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31397ddf-e81c-4a48-a056-f49d22e2bb42 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:00:30.395581 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31397ddf-e81c-4a48-a056-f49d22e2bb42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:00:30.404410 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31397ddf-e81c-4a48-a056-f49d22e2bb42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:00:30.405894 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31397ddf-e81c-4a48-a056-f49d22e2bb42 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:00:30.415140 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1460/2893] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:00:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 00:00:38.157594 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=6 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:00:40.410995 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=6 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:01:44.696243 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:01:44.697673 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:01:44.789163 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d85489-0635-4199-bff5-a2898cc7c403 service nova] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:01:44.794991 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f7aa38-686b-44fa-bd12-6c190c7cffcf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:01:44.796062 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7aa38-686b-44fa-bd12-6c190c7cffcf service placement] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:01:44.796369 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7aa38-686b-44fa-bd12-6c190c7cffcf service placement] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:01:44.796754 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f7aa38-686b-44fa-bd12-6c190c7cffcf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:01:44.798554 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-73f7aa38-686b-44fa-bd12-6c190c7cffcf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:01:44.812201 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7aa38-686b-44fa-bd12-6c190c7cffcf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:01:44.813828 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f7aa38-686b-44fa-bd12-6c190c7cffcf service placement] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:01:44.825507 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1434/2894] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:01:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:01:54.819860 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=37 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:02:33.104934 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52d1e1-92c3-4221-9c40-63646e46f8e0 None None] REQUEST_METHOD: `POST` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:02:33.105462 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52d1e1-92c3-4221-9c40-63646e46f8e0 None None] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:02:33.105615 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52d1e1-92c3-4221-9c40-63646e46f8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:02:33.125694 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.auth.core [None req-3b52d1e1-92c3-4221-9c40-63646e46f8e0 None None] MFA Rules not processed for user `bd94685199f44a218dd96397b02cdc39`. Rule list: `[]` (Enabled: `True`). {{(pid=86604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 00:02:33.126951 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3b52d1e1-92c3-4221-9c40-63646e46f8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:02:33.133899 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1461/2895] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jun 25 00:02:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 00:02:43.118300 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=3 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:04:25.151111 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93112409-e03d-4482-b68a-48cf1bac6e4a None None] REQUEST_METHOD: `POST` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:04:25.151647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93112409-e03d-4482-b68a-48cf1bac6e4a None None] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:04:25.151647 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93112409-e03d-4482-b68a-48cf1bac6e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:04:25.168905 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.auth.core [None req-93112409-e03d-4482-b68a-48cf1bac6e4a None None] MFA Rules not processed for user `bd94685199f44a218dd96397b02cdc39`. Rule list: `[]` (Enabled: `True`). {{(pid=86605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 00:04:25.170119 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-93112409-e03d-4482-b68a-48cf1bac6e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:04:25.178024 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1435/2896] 10.222.0.47 () {58 vars in 957 bytes} [Sat Jun 25 00:04:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 00:04:25.217738 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52d1e1-92c3-4221-9c40-63646e46f8e0 None None] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:04:25.219725 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-3b52d1e1-92c3-4221-9c40-63646e46f8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:04:25.296401 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52d1e1-92c3-4221-9c40-63646e46f8e0 None None] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:04:25.300148 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dec9e1-037d-46ee-9abe-394e811784d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb004ddd208849f6a5316c19e9a916aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:04:25.300828 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dec9e1-037d-46ee-9abe-394e811784d2 service cinder] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:04:25.301118 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dec9e1-037d-46ee-9abe-394e811784d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:04:25.301402 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dec9e1-037d-46ee-9abe-394e811784d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:04:25.303206 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-79dec9e1-037d-46ee-9abe-394e811784d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:04:25.335900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dec9e1-037d-46ee-9abe-394e811784d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:04:25.337374 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dec9e1-037d-46ee-9abe-394e811784d2 service cinder] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:04:25.357090 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1462/2897] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:04:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:04:26.081831 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93112409-e03d-4482-b68a-48cf1bac6e4a None None] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:04:26.102161 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93112409-e03d-4482-b68a-48cf1bac6e4a None None] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:04:26.106015 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4fb73-770e-41a0-8a05-f070f45b964c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:04:26.106678 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4fb73-770e-41a0-8a05-f070f45b964c service neutron] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:04:26.106957 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4fb73-770e-41a0-8a05-f070f45b964c service neutron] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:04:26.107229 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4fb73-770e-41a0-8a05-f070f45b964c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:04:26.109085 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-02d4fb73-770e-41a0-8a05-f070f45b964c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:04:26.122745 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4fb73-770e-41a0-8a05-f070f45b964c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:04:26.124147 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4fb73-770e-41a0-8a05-f070f45b964c service neutron] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:04:26.135612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1436/2898] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:04:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:04:31.822044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dec9e1-037d-46ee-9abe-394e811784d2 service cinder] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:04:31.833562 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dec9e1-037d-46ee-9abe-394e811784d2 service cinder] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:04:31.837284 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5151ced-e99c-4d1b-be89-cce6b3aeeda3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:04:31.837982 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5151ced-e99c-4d1b-be89-cce6b3aeeda3 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:04:31.838265 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5151ced-e99c-4d1b-be89-cce6b3aeeda3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:04:31.838543 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5151ced-e99c-4d1b-be89-cce6b3aeeda3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:04:31.840375 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-e5151ced-e99c-4d1b-be89-cce6b3aeeda3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:04:31.865820 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5151ced-e99c-4d1b-be89-cce6b3aeeda3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:04:31.867392 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5151ced-e99c-4d1b-be89-cce6b3aeeda3 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:04:31.877431 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1463/2899] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:04:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:04:36.130793 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=13 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:04:41.873391 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=58 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:04:47.506539 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4fb73-770e-41a0-8a05-f070f45b964c service neutron] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:04:47.519775 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4fb73-770e-41a0-8a05-f070f45b964c service neutron] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:04:47.526584 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bde6c4-3e91-4a7c-9c69-6e5227844236 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:04:47.527248 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bde6c4-3e91-4a7c-9c69-6e5227844236 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:04:47.527738 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bde6c4-3e91-4a7c-9c69-6e5227844236 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:04:47.528012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bde6c4-3e91-4a7c-9c69-6e5227844236 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:04:47.531891 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-49bde6c4-3e91-4a7c-9c69-6e5227844236 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:04:47.545813 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bde6c4-3e91-4a7c-9c69-6e5227844236 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:04:47.547498 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bde6c4-3e91-4a7c-9c69-6e5227844236 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:04:47.559785 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1437/2900] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:04:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:04:57.554838 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=6 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:06:14.967759 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5151ced-e99c-4d1b-be89-cce6b3aeeda3 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:14.976842 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5151ced-e99c-4d1b-be89-cce6b3aeeda3 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:14.980611 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b77996-58d7-4e38-b2d3-338a26ff79c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:14.981254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b77996-58d7-4e38-b2d3-338a26ff79c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:14.981533 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b77996-58d7-4e38-b2d3-338a26ff79c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:14.981804 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b77996-58d7-4e38-b2d3-338a26ff79c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:14.990907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b77996-58d7-4e38-b2d3-338a26ff79c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:14.992334 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b77996-58d7-4e38-b2d3-338a26ff79c3 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:15.001177 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1464/2901] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:06:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:06:15.843346 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bde6c4-3e91-4a7c-9c69-6e5227844236 service nova] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:15.857911 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bde6c4-3e91-4a7c-9c69-6e5227844236 service nova] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:15.864437 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38e45-4e8a-4930-9687-0b96c6c9a107 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feb9bec9f1894e179e24a14b81b3f23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:15.865199 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38e45-4e8a-4930-9687-0b96c6c9a107 service nova] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:15.865480 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38e45-4e8a-4930-9687-0b96c6c9a107 service nova] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:15.865752 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38e45-4e8a-4930-9687-0b96c6c9a107 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:15.867834 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-3ed38e45-4e8a-4930-9687-0b96c6c9a107 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:06:15.882479 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38e45-4e8a-4930-9687-0b96c6c9a107 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:15.884140 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38e45-4e8a-4930-9687-0b96c6c9a107 service nova] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:15.896729 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1438/2902] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:06:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:06:24.997190 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=6 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:06:25.891138 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=6 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:06:28.292909 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b77996-58d7-4e38-b2d3-338a26ff79c3 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:28.302619 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b77996-58d7-4e38-b2d3-338a26ff79c3 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:28.306331 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24eef8-fb00-403c-9906-8e7d4f5f5c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e38a311f2b614a3598f6b51b3a235178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:28.306966 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24eef8-fb00-403c-9906-8e7d4f5f5c35 service neutron] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:28.307249 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24eef8-fb00-403c-9906-8e7d4f5f5c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:28.307527 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24eef8-fb00-403c-9906-8e7d4f5f5c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:28.376722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24eef8-fb00-403c-9906-8e7d4f5f5c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:28.378212 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24eef8-fb00-403c-9906-8e7d4f5f5c35 service neutron] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:28.397407 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1465/2903] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:06:31.245094 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38e45-4e8a-4930-9687-0b96c6c9a107 service nova] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:31.262565 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38e45-4e8a-4930-9687-0b96c6c9a107 service nova] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:31.269845 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e02ef-65a9-4445-8b68-346b3f7809e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:31.270612 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e02ef-65a9-4445-8b68-346b3f7809e0 service placement] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:31.270900 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e02ef-65a9-4445-8b68-346b3f7809e0 service placement] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:31.271176 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e02ef-65a9-4445-8b68-346b3f7809e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:31.273013 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-c41e02ef-65a9-4445-8b68-346b3f7809e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:06:31.287012 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e02ef-65a9-4445-8b68-346b3f7809e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:31.288955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e02ef-65a9-4445-8b68-346b3f7809e0 service placement] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:31.302457 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1439/2904] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:06:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:06:34.365386 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24eef8-fb00-403c-9906-8e7d4f5f5c35 service neutron] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:34.395915 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24eef8-fb00-403c-9906-8e7d4f5f5c35 service neutron] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:34.408298 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac00c8-ce87-4e2b-aeba-0104e11999ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:34.409385 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac00c8-ce87-4e2b-aeba-0104e11999ac service placement] REQUEST_METHOD: `GET` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:34.409709 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac00c8-ce87-4e2b-aeba-0104e11999ac service placement] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:34.410044 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac00c8-ce87-4e2b-aeba-0104e11999ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:34.412984 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-d1ac00c8-ce87-4e2b-aeba-0104e11999ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:06:34.429209 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e02ef-65a9-4445-8b68-346b3f7809e0 service placement] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:34.445664 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e02ef-65a9-4445-8b68-346b3f7809e0 service placement] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:34.452306 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:34.452852 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac00c8-ce87-4e2b-aeba-0104e11999ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:34.453433 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:34.453815 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:34.454453 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] PATH_INFO: `/v3/users/bd94685199f44a218dd96397b02cdc39` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:34.454651 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac00c8-ce87-4e2b-aeba-0104e11999ac service placement] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:34.456415 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd94685199f44a218dd96397b02cdc39)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:34.458725 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:34.466722 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1466/2905] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:06:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:06:34.525028 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd94685199f44a218dd96397b02cdc39'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.537308 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd94685199f44a218dd96397b02cdc39'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.547955 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd94685199f44a218dd96397b02cdc39'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.548697 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd94685199f44a218dd96397b02cdc39'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.557850 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd94685199f44a218dd96397b02cdc39'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.565887 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1440/2906] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jun 25 00:06:34 2022] DELETE /identity/v3/users/bd94685199f44a218dd96397b02cdc39 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 00:06:34.750346 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac00c8-ce87-4e2b-aeba-0104e11999ac service placement] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:34.762055 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac00c8-ce87-4e2b-aeba-0104e11999ac service placement] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:34.827389 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:34.828252 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:34.828532 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:34.828841 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] PATH_INFO: `/v3/projects/f10b8a1bc1654ae4ac5c08c23ca9e399` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:34.830008 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10b8a1bc1654ae4ac5c08c23ca9e399)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:34.831907 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:34.878772 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10b8a1bc1654ae4ac5c08c23ca9e399', 'request_id': 'req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.894476 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10b8a1bc1654ae4ac5c08c23ca9e399', 'request_id': 'req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.901908 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10b8a1bc1654ae4ac5c08c23ca9e399', 'request_id': 'req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.910868 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:06:34.938254 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] The token cache is being invalidate because project f10b8a1bc1654ae4ac5c08c23ca9e399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 00:06:34.938535 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.939162 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:34.940748 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1467/2907] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jun 25 00:06:34 2022] DELETE /identity/v3/projects/f10b8a1bc1654ae4ac5c08c23ca9e399 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 00:06:34.944996 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:34.946393 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:06:34.959538 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221e980-0085-463d-8389-03d95db68441 admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:35.022301 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:35.023628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:35.024284 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:35.024570 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] PATH_INFO: `/v3/users/7c980082abfe4710ab6a21cd0d827808` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:35.026049 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c980082abfe4710ab6a21cd0d827808)` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:35.027982 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:35.079132 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c980082abfe4710ab6a21cd0d827808'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.089745 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:06:35.092292 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c980082abfe4710ab6a21cd0d827808'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.101844 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c980082abfe4710ab6a21cd0d827808'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.102724 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c980082abfe4710ab6a21cd0d827808'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.109521 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.notifications [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c980082abfe4710ab6a21cd0d827808'} {{(pid=86605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.115343 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1441/2908] 10.222.0.47 () {58 vars in 1223 bytes} [Sat Jun 25 00:06:34 2022] DELETE /identity/v3/users/7c980082abfe4710ab6a21cd0d827808 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 00:06:35.298856 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] Authenticating user token {{(pid=86604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:35.300368 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.fernet_utils [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:06:35.313579 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdf697-f15e-4eb1-8cfd-9c5fb54dc3e8 admin admin] Validating token access rules against request {{(pid=86604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:35.371900 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19e6b8d9d8f94b0c8f316b8b2060e76d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1b6f1fb3ca4d42909a61ac0a3da08290', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:35.372794 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:35.373084 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:35.373361 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] PATH_INFO: `/v3/projects/4a9c7101421e47999b953bfa8e08d5a3` {{(pid=86604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:35.374540 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a9c7101421e47999b953bfa8e08d5a3)` {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:35.376456 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] RBAC: Authorization granted {{(pid=86604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:35.422552 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a9c7101421e47999b953bfa8e08d5a3', 'request_id': 'req-9500cb88-c703-4661-b8d2-4219b11fa2c3'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.430518 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a9c7101421e47999b953bfa8e08d5a3', 'request_id': 'req-9500cb88-c703-4661-b8d2-4219b11fa2c3'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.436768 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a9c7101421e47999b953bfa8e08d5a3', 'request_id': 'req-9500cb88-c703-4661-b8d2-4219b11fa2c3'} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.447781 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] The token cache is being invalidate because project 4a9c7101421e47999b953bfa8e08d5a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 00:06:35.448069 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.448736 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG keystone.notifications [None req-9500cb88-c703-4661-b8d2-4219b11fa2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 00:06:35.450256 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: [pid: 86604|app: 0|req: 1468/2909] 10.222.0.47 () {58 vars in 1232 bytes} [Sat Jun 25 00:06:35 2022] DELETE /identity/v3/projects/4a9c7101421e47999b953bfa8e08d5a3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 00:06:45.111862 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:06:45.444848 dv-cinder-847624-1-iscsi devstack@keystone.service[86604]: DEBUG dbcounter [-] [86604] Writing DB stats keystone:DELETE=8,keystone:SELECT=58,keystone:INSERT=1 {{(pid=86604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 00:06:46.564233 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] Authenticating user token {{(pid=86605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 00:06:46.565704 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:06:46.633790 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb1a5b-1e7d-442b-84a6-b04eb303bfad admin admin] Validating token access rules against request {{(pid=86605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 00:06:46.691271 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af42a21-9a6b-45f2-b14a-a041b5a34f12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a243d444b7ae487cb0331523d4404d19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9a1b392b7034790a51237458cab8b75', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 00:06:46.692379 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af42a21-9a6b-45f2-b14a-a041b5a34f12 service placement] REQUEST_METHOD: `GET` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 00:06:46.692689 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af42a21-9a6b-45f2-b14a-a041b5a34f12 service placement] SCRIPT_NAME: `/identity` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 00:06:46.692968 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af42a21-9a6b-45f2-b14a-a041b5a34f12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 00:06:46.694736 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.fernet_utils [None req-8af42a21-9a6b-45f2-b14a-a041b5a34f12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 00:06:46.705692 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af42a21-9a6b-45f2-b14a-a041b5a34f12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 00:06:46.707866 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af42a21-9a6b-45f2-b14a-a041b5a34f12 service placement] RBAC: Authorization granted {{(pid=86605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 00:06:46.717737 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: [pid: 86605|app: 0|req: 1442/2910] 10.222.0.47 () {60 vars in 1365 bytes} [Sat Jun 25 00:06:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 00:06:56.713628 dv-cinder-847624-1-iscsi devstack@keystone.service[86605]: DEBUG dbcounter [-] [86605] Writing DB stats keystone:SELECT=58 {{(pid=86605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}